Yocto Build Failure Swat Team

2020-11-10T15:45:54Z

Kergoth:

== Overview ==

The role of the SWAT team is to monitor the autobuilder and investigate all failures to ensure that they are logged and brought to the attention of the appropriate owner.

All builds that are run on the [https://autobuilder.yoctoproject.org public autobuilder] are important for the Yocto Project, whether they be a post-merge validation run (for master or a release branch) or a pre-merge test build (for master-next, ross/mut, stable/* and others). SWAT doesn't cover company-internal autobuilders.

Every build should be monitored by the SWAT team unless the [[BuildLog]] entry for that build indicates otherwise. SWAT is opt-out by whomever triggers a build on the autobuilder, not opt-in.

SWAT isn't responsible for resolving any issues encountered on the autobuilder. Their focus is on performing minimal analysis of a failure in order to ensure that it is logged and brought to the attention of a appropriate owner, a process we'll refer to as pre-triage.

== Roles ==

Active member: the currently active member of the SWAT team is expected to monitor the autobuilder and pre-triage failures in a timely fashion. Team members are active for one week at a time and rotation is usually simple round robin through the members list. If for some reason the next person isn't available then they can be defer a week until they are.

SWAT Chair: the SWAT chair provides backup cover for the active member and is a first point of contact for SWAT. [[User:RossBurton | Ross Burton]], Armin Kuster and Richard Purdie are the SWAT Chair people.

SWAT Facilitator: the SWAT facilitator is responsible for managing the rotation process. [[User:Stephen_K._Jolley | Stephen K. Jolley]] is the current SWAT Facilitator.

== Process ==

The [[BuildLog]] wiki page is automatically updated by the autobuilder when a new build is triggered. This often includes guidance for SWAT (the "Reason" field when triggering the build) such as to ignore it (if the build is expected to break and the owner is monitoring themselves) or a request to report failures straight to a specific person (if the build only contains changes from a single person, such as a GCC upgrade). Unless when told otherwise, the usual process is as follows:

For builds against master or a release branch, any issues observed should be [[#Filing_bugs | filed in Bugzilla]].

For builds against other branches (master-next, ross/mut, -next branches for stable releases, etc.), attempt to identify what patch in the branch is likely responsible for the failure. For example, if <tt>wget</tt> fails with <tt>libgnutls</tt> errors and there is a GnuTLS upgrade in the branch, that is the likely candidate. If a patch can be identified that isn't also in master or release branch, reply on the mailing list with the failure details. If it isn't obvious which patch is responsible for the failure then [[#Filing_bugs | file a bug]] and ensure the branch owner is either the assignee or on the CC list.

'''If in doubt, file a bug'''. All observed errors must be actioned unless a patch has already been sent for the issue, in which case please make note of this in the [[BuildLog]].

If the issue is in the infrastructure or autobuilder itself then file a bug against , infrastructure bugs should be assigned to [[User:Halstead| Michael Halstead]] and autobuilder logic bugs to [[User:Rpurdie | Richard Purdie]].

The results of pre-triage for an issue should be added to the corresponding entry in the [[BuildLog]], including a link to the resolution (patch name, bug link, etc) and a brief summary of the issue. Every issue should be added to the build log so it acts as a build status report.

Note that some builds, in particular the "perf" builds are not listed on [[BuildLog]] unless the build fails (to try and reduce noise on the log). Failures in performance test builds should be handled like any other build.

The net result is all failures listed in [[BuildLog]] should have outcomes listed against them from the person on SWAT at the time.

=== Filing bugs ===

When filing the bug two things are required:
* A link to the build failure. This can either be the build page (such as https://autobuilder.yoctoproject.org/typhoon/#/builders/34/builds/168) or the [http://errors.yoctoproject.org/Errors/ error reports] page (such as http://errors.yoctoproject.org/Errors/Details/199667/).
* The error itself. Trim the log down to just the error and any relevant context in the bug description, and also attach the complete log file. Autobuilder logs are not persistent so the full log must be attached.

=== Process summary ===

* Monitor builds via one or more of:
** the autobuilder: [https://autobuilder.yoctoproject.org/typhoon/ main page], [https://autobuilder.yoctoproject.org/typhoon/#/console console view]. Orange builds have warnings, red builds failed with errors.
** the [http://errors.yoctoproject.org/Errors/ Error Reports] web page which has an entry for each recipe that fails. A link to the appropriate search is on the autobuilder's Console View.
** the [[BuildLog]] wiki page. This has links to each build log that wasn't successful.
** the [https://lists.yoctoproject.org/listinfo/yocto-builds yocto-builds] mailing list. This gets a mail for every unsuccessful build.
* Pre-triage each failure:
** File a bug ''or'' respond to a patch ''or'' note known issues
** Update the [[BuildLog]] with the result of pre-triage, linking to the resolution where possible.

== Questions / Contact ==

If you have queries about the SWAT process you may reach out to the SWAT Facilitator [[User:Stephen_K._Jolley | Stephen K. Jolley]] or the SWAT Chair [[User:RossBurton | Ross Burton]].

== Members ==

[[User:Leonardo_Sandoval | Leo Sandoval]]

[[User:SaulWold | Saul Wold]]

[[User:PaulEggleton | Paul Eggleton]]

[[User:Naveen Kumar Saini | Naveen Saini]]

Armin Kuster (place me anywhere)

[[User:Kergoth | Christopher Larson]]

LesserKnownFeatures

2012-08-23T00:43:17Z

Kergoth: Initial creation

Let’s go over a few Yocto/OE capabilities which seem to be less commonly known.

Variable Typing
---------------

The first item I will cover is the "Variable Typing" support. This feature allows one to declare the 'type' of a given metadata variable. This both enables a type check to ensure that it's syntactically valid, and returns an object of the appropriate type when using the python interface. Admittedly, this is promotion of something I created, but I do believe there's value here, and future posts will cover a variety of features created by a variety of individuals.

The implementation in oe-core consists of two components:

* Type creation python modules, whose job it is to construct objects of the defined type for a given variable in the metadata
* typecheck.bbclass, which iterates over all configuration variables with a type defined and uses oe.types to check the validity of the values

This gives us a few benefits:

* Automatic sanity checking of all configuration variables with a defined type
* Consolidates the logic surrounding what to do with the value of a given variable. For variables like PATH, this is simply a split(), but for boolean variables, any duplication can result in unclear semantics. For example, we may not know whether our choices are 1/0, empty/nonempty, true/false, etc.
* Make it easier to create a configuration UI, as the type information could be used to provide a better interface than a text edit box (e.g checkbox for 'boolean', dropdown for 'choice')

To enable configuration time type checking of the config variables which have the appropriate metadata, set the following: `INHERIT += "typecheck"`. This is done by the 'mel' distro automatically.

List of currently available types: list, choice, regex, boolean, integer, float

An example of the appropriate metadata with failing typecheck:

BAZ = "foo"
BAZ[type] = "boolean"

$ bitbake -p
FATAL: BAZ: Invalid boolean value 'foo'

Further examples of metadata values:

FOO = "alpha"
FOO[type] = "choice"
FOO[choices] = "alpha beta theta"

PACKAGES[type] = "list"

LIBTOOL_HAS_SYSROOT = "yes"
LIBTOOL_HAS_SYSROOT[type] = "boolean"

PATH[type] = "list"
PATH[separator] = ":"

Examples of usage of these from python:

python () {
import oe.data
for pkg in oe.data.typed_value("PACKAGES", d):
bb.note("package: %s" % pkg)

for path in oe.data.typed_value("PATH", d):
bb.note("PATH element: %s" % path)

assert(oe.data.typed_value("FOO", d) == "alpha")
assert(oe.data.typed_value("LIBTOOL_HAS_SYSROOT", d) == True)
}

OE python package
-----------------

The OE python package has a number of modules which provide some valuable pieces of functionality. Please, take the time to look over these modules before writing any python in the metadata, as they can prevent unnecessary reinvention of the wheel.

Examples (random selection of bits from the oe package):

* oe.path.relative - return a relative path from src to dest
* oe.path.copytree - recursively copy a tree from src to dest
* oe.path.find - acts like the ‘find’ command, traversing a tree and yielding the full paths to the files
* oe.utils.inherits - return True if the metadata inherits from any of the specified classes

Classes
-------

* externalsrc

This class is used to build from an existing local source tree, rather than the usual fetch/unpack/patch process. This is an invaluable tool for active development.

* gitver/gitpkgv

gitpkgv is used to extract the version from a git repository referenced from SRC_URI, and use it in the package versioning.
gitver is used for the same purpose, but to extract from a local repository, rather than a bitbake fetched repository

* lib_package

This class is used to split out the binaries from the main library for recipes whose primary artifact is the library

* recipe_sanity (used via INHERIT)

This class is used to run a set of basic sanity checks against the metadata of a recipe. It checks for some common user errors.

* archiver (used via INHERIT)

This class is used to archive sources/patches/logs for a recipe, usually for license compliance. It is quite configurable, to cover the common use cases, and ensure it can meet the needs of any legal department.

Yocto Project - User contributions [en]

Yocto Build Failure Swat Team

LesserKnownFeatures