Patchtest
Patchtest
Patchtest is a tool designed for improving the quality of Yocto Project and OpenEmbedded contributors' patch submissions, while simultaneously reducing the level of review effort required from project maintainers. It does this by providing a set of scripts that users can test their patches with prior to submission ("host" mode), and which can be used as part of an automated service that provides periodic feedback for patches received by Patchwork ("guest" mode).
Using Patchtest
See the patchtest README for instructions on how to use patchtest in your local workspace, and the meta-patchtest README for details on automated/guest mode setup.
Patchtest "FAIL:" Descriptions
The patchtest feedback email is designed to provide concise instructions when one or more of the tests has failed, but additional information on specific failures is provided below:
LIC_FILES_CHKSUM changed without "License-Update:" tag and description in commit message
Patchtest checks to ensure that any changes made to LIC_FILES_CHKSUM lines (including checksums for new files) are accounted for in the commit message with at least one "License-Update: <reason>" tag. If this is not present, then this test will fail.
A CVE tag should be provided in the commit message with format: "CVE: CVE-YYYY-XXXX"
Whenever a submission adds a CVE patch, patchtest will check both the patch and the parent mbox's commit message for a "CVE: CVE-YYYY-XXXX" tag. While this has historically not been required, including the tag in both locations aids review and maintenance.
Please include a commit message on your patch explaining the change
This response is received whenever a submission has an empty commit message.
Edit shortlog so that it is 90 characters or less (currently X characters)
This failure indicates that the shortlog (i.e. the subject line) for the patch is so long that it may not be easily readable on some mail clients. Some subject prefixes (e.g. the branch name when submitting a patch for an older release) may unavoidably inflate the length, but the shortlog should be kept as short and concise as possible so that it is clear what the patch does and what it changes. Extra information can always be provided in the commit message.
Invalid author auh@auh.yoctoproject.org. Resend the series with a valid patch author
A patch authored by the Auto-Upgrade Helper (AUH) was detected. These patches are meant to provide maintainers with a sanity check for recipe upgrades and a starting point for testing them, so patches submitted with that authorship are marked as failures to indicate that they have not been fully validated.
Tests failed for the patch, but the results log could not be processed due to excessive result line length.
The patchtest-send-results script checks the test results' maximum line length to help ensure that no unintended and/or malicious code is sent to the mailing list with the results. If this message is present, the user should re-test their patch(es) locally using the patchtest script in openembedded-core/scripts before further escalation. If the issue persists after re-testing and re-submission it should be reported on the Yocto Project Bugzilla under the "Yocto Project Subprojects" -> "Patchtest" category with a "Major" priority (include the patch file as an attachment).
Submitting Bug Reports
If you suspect there is an issue with the patchtest feedback emails, the service itself, or any of the documentation, please submit an issue on the Yocto Project Bugzilla Bugzilla under the "Yocto Project Subprojects" -> "Patchtest" category.
Patchtest Architecture Notes
| Test Name | Category | Uses Tinfoil | Notes |
|---|---|---|---|
| test_upstream_status_presence_format | TestPatch | No | |
| test_signed_off_by_presence | TestPatch | No | |
| test_cve_tag_format | TestPatch | No | |
| test_signed_off_by_presence | TestMbox | No | |
| test_shortlog_format | TestMbox | No | |
| test_shortlog_length | TestMbox | No | |
| test_series_merge_on_head | TestMbox | No | |
| test_target_mailing_list | TestMbox | No | |
| test_mbox_format | TestMbox | No | |
| test_commit_message_presence | TestMbox | No | |
| test_bugzilla_entry_format | TestMbox | No | |
| test_author_valid | TestMbox | No | |
| test_non_auh_upgrade | TestMbox | No | |
| test_pylint | PyLint | No | |
| test_license_presence | TestMetadata | Yes | |
| test_lic_files_chksum_presence | TestMetadata | Yes | |
| test_lic_files_chksum_modified_not_mentioned | TestMetadata | No | |
| test_max_line_length | TestMetadata | Example | |
| test_src_uri_left_files | TestMetadata | Yes | Has a linked pretest: pretest_src_uri_left_files |
| test_summary_presence | TestMetadata | Yes |
