User:MichaelHalstead

From Yocto Project
Jump to: navigation, search

Testing watchlists again and again and again and again and again

IDPStatusSeverityProductSummary (218 tasks)  
11467*
11467

Right now, meta-swupd is optional (and not tested) and OSTree support is experimental. We need to pick one of these two and integrate it: - recipes part of intel-iot-refkit (directly or as submodule)

- automated testing
HighIN PROGRESS IMPLEMENTATIONenhancementIoT Reference OS Kitintegrate a system update solution 
11452*
11452

I see this is only further creating confusion.

Couldn't we just add flags to the layer index showing compatible status, testing status etc?

It would completely suck having to search the layer index to find a recipe, then having to move to another layer index to check compatible and testing status.
HighNEWenhancementLayer IndexCreate a layers.yoctoproject.org site 
11707*
11707

The cron job that runs patchtest sometimes (twice this year) is unable to test incoming patches because some incoming data (patches?) is causing the share folder to get corrupted. The first time this happened [1], the git commands under the latter folder could not be done, indicating corruption on the .git folder, and the second one [1], someone the credentials were removed.


[1] first series untested by patchtest the first time this year https://patchwork.openembedded.org/series/6725/

[2] first series untested by patchtest the second time this year

https://patchwork.openembedded.org/series/7405/
HighNEWnormalPatchwork/Patchtestopenembedded-core share folder between host and guest get corrupted 
11140*
11140 The fix is now implemented in OE-Patchwork. Thanks to Michael for the quick response.
HighRESOLVEDmajorPatchwork/Patchtestpatchwork: Web UI throws an error instead of series view when user is not authenticated 
11469*
11469 IoT RefKit should include enablers for using core functionality of both Wayland/Weston and X11/OpenGL for graphics output.
HighRESOLVEDenhancementIoT Reference OS KitGraphics support for IoT Reference Kit 
11475*
11475 Implement an interactive, graphics-based demo for a simple rock-paper-scissors demo. Use a camera (RealSense camera or a normal web camera) to detect the hand sign {rock, paper, scissors} that the user does. Match that against a random play made by the computer. Indicate on the screen how the platform has identified the hand sign (for example using a bounding box).
LowIN PROGRESS IMPLEMENTATIONenhancementIoT Reference OS KitRock-paper-scissors demo based on RefKit computer vision profile 
11142*
11142

Several of the tests in oe-selftest invoke bitbake with recipe targets, in many cases building relatively complex recipes with non-trivial dependency chains.

For several tests fetching and building isn't strictly necessary and the needs of the test could be as easily met with an artificial recipe in meta-selftest which uses a local file as its source, one immediate example which prompted this report is oescripts.TestScripts.test_cleanup_workdir

We should review the targets bitbake is invoked with in oe-selftest and aim to:

  • use artificial recipes, with a local file source, where possible
  • reduce reliance on recipes being present in oe-core, i.e. where practical oe-selftest should use recipes contained in meta-selftest to ensure the results are easily repeatable and independent of unrelated changes to oe-core i.e. the recent (at time of reporting) removal of GPLv2 recipes.
LowNEWenhancementAutomated Build TestingUse artificial recipes for oe-selftest where possible to reduce work 
11304*
11304

(In reply to comment #3) > The reason I set this to the future is because of the uncertainty that swupd > will be the official update feature supported by Yocto now and in the > future. If what I think I know is not correct then I can work on this for > 2.4.

See comment #1. I'd like to use this issue to track the base class for image update testing and bug #9783 for the swupd-specific part. Following that logic, this bug here is needed for 2.4 - the sooner, the better.
LowNEWnormalTest Plans/SuiteCreate automated testcases for system update 
11480*
11480 Integrate YOLO (https://pjreddie.com/darknet/yolo/) to Refkit for real-time object detection.
LowNEWenhancementIoT Reference OS KitIntegrate YOLO to Refkit 
11499*
11499 Check what is missing from computer vision profile to enable Speaker Recognition demo and integrate those. At minimum also include documentation for demo setup, possibly also add audio and classification front ends in Python.
LowNEWenhancementIoT Reference OS KitSpeaker recognition neural network demo 
11670*
11670

When we use BBCLASSEXTEND, we create "variants" (a term used in a number of different places) of the recipe, however in the file path namespace they end up as "virtual:...". This probably wouldn't be so bad except that "virtual/..." is a commonly used naming convention for virtual targets and is completely unrelated to this, which could be confusing.

We could simply rename the virtual: prefix - it's mostly internal to bitbake, although it is printed out on the console and in logs.
LowNEWenhancementBitBakeImprove terminology around BBCLASSEXTEND 
11463*
11463 The PR got merged to master.
LowRESOLVEDenhancementIoT Reference OS KitRefKit: Python version alignment 
10050*
10050

For us , we need this function to generate logs for this case:

One oe runner static *Total* case includes many real cases , so to the runner, it just only generate one result for those many real cases in the log file, which doesn't reflect the real results.
MediumACCEPTEDenhancementAutomated Runtime Testingoeqa runner generate wrong test results in log file when running dynamic test cases 
11612*
11612 Create a suite to test support for building targets with multiple configurations (multiconfig). The corresponding section on the YP Mega Manual is http://www.yoctoproject.org/docs/2.3/mega-manual/mega-manual.html#platdev-building-targets-with-multiple-configurations
MediumACCEPTEDnormalAutomated Build TestingQA Bitbake: Create tests for multiconfig support 
11255*
11255 Thanks Stephen, I've corrected the status.
MediumIN PROGRESS DESIGNenhancementIoT Reference OS Kit TestingQA: Clean up and enablement of single-node Bluetooth test cases on meta-iotqa 
11510*
11510

This will broken up into three bugs:

This one: Will create a new manifest file (JSON), on which all packages specify what modules and files they require, this is basically just a replacement of the manifest we use at the moment with a more organized one, since it will include a script to "convert" the new manifest to the old one so bitbake can still parse it the same way.

Second bug/enchancement: This one will handle the part on which packages are parsed, this should be done via python, by reading the JSON manifest file directly, and modifying bitbake's dictionary on the go.

Up to this point, the manifest replacement will be complete, but other than it being more organized and having got rid of the script to create it (confusing), it will still have the same other problems stated above.

Third bug/enhancement:

This one will create a test which may be run on top of testimage, that will check every python package to see specifically which modules it needs, and will create a manifest file (JSON) automatically which can be used to replace the current file, this can be run either on every release or when adding a new package if one is lazy enough to not check the modules the new package needs to work.
MediumIN PROGRESS DESIGNenhancementOE-CoreImprove the way we handle python's packaging (autopackaging) 
11694*
11694

Once we merge the new python manifest file using a JSON format, it should be parsed automatically by bitbake, as opposed to having a script which translates from JSON to bash.

This poses a few complications since parsing times are different from bash to python, and bitbake won't know which python packages exists until it reads the JSON file, which may lead to some unusual behavior.

Up to this point, the manifest replacement will be complete, and we should be able to get rid of the script thats being used to create the manifest everytime.
MediumIN PROGRESS DESIGNenhancementOE-CorePython JSON manifest should be parsed directly by bitbake 
11695*
11695 Not sure why this needs to use testimage. If we assume that native python is the same version as target python then we can introspect the native python when that is built.
MediumIN PROGRESS DESIGNenhancementAutomated Runtime TestingAdd test to create python manifest(s) automatically 
10535*
10535 Our current workflow does not require this functionality, so move it to Future.
MediumIN PROGRESS IMPLEMENTATIONenhancementCROPSBuild container for Maven/Tycho build for Eclipse CROPS Plugin 
11537*
11537

RMC should provide an option to return the fingerprint of the current board in MD5 format.

Example:

./rmc --fp

will return b2c1a220edd348efb0108e8c98e9a28d .
MediumIN PROGRESS IMPLEMENTATIONnormalBSPsRMC: Get board fingerprint 
11478*MediumIN PROGRESS REVIEWenhancementIoT Reference OS KitMove caffe from meta-refkit-extra to meta-refkit 
11589*
11589

Please add below configuration in conf/local.conf and the build will successful. MKUBIFS_ARGS = " -m 512 -e 15360 -c 3600 " UBINIZE_ARGS = " -p 16KiB -m 512 -s 512 "


https://lists.yoctoproject.org/pipermail/yocto/2013-June/014757.html

Enhancement on image_type.bbclass in order to pre-early bring out the error and example configuration for any ubi and ubifs image build.
MediumIN PROGRESS REVIEWnormalOE-Coreubi and ubifs image compression failed on do_image_ubi and do_image_ubifs[1]
11590*MediumIN PROGRESS REVIEWnormalOE-Corecreating root file systems using elf failed applying patch fix-makefile-to-find-libz.patch[2]
11616*
11616 do we still have this failure
MediumIN PROGRESS REVIEWnormalOE-CoreMesa build failed with gcc 7 (ARM) 
11675*
11675

Leonardo - thanks, I applied the change to the 2.3.1 bitbake manual. See steps 5 and 9 of http://www.yoctoproject.org/docs/2.3.1/bitbake-user-manual/bitbake-user-manual.html#the-hello-world-example

Once I hear from Richard that he's merged it into bitbake, I will mark this as RESOLVED.

Kristi
MediumIN PROGRESS REVIEWnormalBitBake User Manualbitbake.conf URL link example should be updated 
11684*
11684

There are three main API deprecations in Django-1.10, and the warnings start to appear in Django 1.9.

In order to support forward migration, these API changes need to be addressed.
MediumIN PROGRESS REVIEWnormalToasteraddress RemovedInDjango110Warning's for Django >= 1.9 
8919*
8919 This is low priority for 2.3 and there is no window time to work on this area during 2.3.
MediumNEWenhancementOE-CoreCode coverage: add ability to create coverage-based test exclusion reports 
8921*
8921

(In reply to comment #3) > As this issue got assigned to me I'd like to know why do we need this and is > this functionality can be implemented using python-coverage?

I moved to 2.4 and for some reason it got assigned to you.

For your question, oe-selftest is already using python-coverage (version 4.x), so we can produce data. The main task here is the analysis of this data. No plans to do the later during 2.3
MediumNEWenhancementOE-CoreCode coverage: add ability to create missing 'coverage area' reports 
9769*
9769 Will achieve this same request via buildsets that run on a specific set of workers.
MediumNEWenhancementAutoBuilder[Autobuilider] Ability to select the worker for a forced build 
10308*
10308

As noted in the mailing list.

The problem is that apparently a previous change activated some normally deactivated code to allow signature by-pass.

The previous code was implemented in reverse, i.e. signature/digest validation was always deactivated instead of always activated.

This permits a corrupted package to avoid validation and install.


As noted on the mailing list by Hongxu, the fix for this issue would be to correct the validation arguments, so the default is enabled, and the --no* options disable the default.


Alternatively we disable the previous patch and go back to the community setting avoiding this chunk of code.
MediumNEWnormalOE-Corerpm installs bad signature package with option --nodigest --nosignature failed 
10537MediumNEWenhancementCROPSUse "new project template" framework to create sample Make project 
10538MediumNEWenhancementCROPSUse "new project template" framework to create sample Automake project 
10540MediumNEWenhancementCROPSUse "new project template" framework to create sample CMake project 
10552MediumNEWenhancementCROPSDevtool add from existing Eclipse project within the Eclipse CROPS plugin 
10553*
10553 This is intended to support the recipes brought in by devtool.
MediumNEWenhancementCROPSContext sensitive recipe syntax support within the Eclipse CROPS plugin 
10555MediumNEWenhancementCROPSDevtool upgrade from existing recipe into an Eclipse project within the Eclipse CROPS plugin 
10632*
10632

The Layer Index provides a list of available distros, but Toaster does not take advantage of that information.

The proposal is to:

 (a) Add a selection box with lookahead, just like the MACHINE selection.
 (b) Add a selection Toaster Table, again like the MACHINE selection.
 (c) Leave the existing manual edit in the configure variables page, in case the user wants to set a distro not currently published in the Index.
I have a working implementation which I plan to contribute to Toaster.
MediumNEWenhancementToasterAdd distro selection support 
10847*
10847

(In reply to comment #3) > Yes, this makes sense. Thanks. > > However, I'm still a bit concerned with the potential complexity of this > setup. Would it be better to use serial interface? What exactly setting, > running and maintaining dhcp infrastructure would give us? Would it mean > that to run test suite on my machine I'd need to also setup dhcpd > infrastructure? Agreed, as I was filing this bug and finding out the implications of it, I realized the complexity of it.

I commented on bug 10833 that adapting the serial interface procedure to the automation would be a way to solve such problem.

Still, dynamic IP addresses for test targets would be a consolidated solution that might be worth looking for future releases.
MediumNEWenhancementAutomated Runtime Testingtestimage: add the ability to test dynamic IP addr targets 
10947*
10947

Toaster QA team has to review the erased automated testcases located in

toaster_automation_tests.py to determine which testcases are still valid and fix the coding of them.
MediumNEWnormalToasterReintegrate fully functional and corrected toaster automation testcases from toaster_automation_tests.py 
11145*
11145

It would make sense to be able to optionally start a PRServer with the controller when the autobuilder is configured for a single shared PRServer running on the controller, a setup we'd like to switch to on the project's autobuilder clusters.

Currently we automatically start a shared local PRServer (with an up-to-date clone of bitbake) on each worker when PRSERV_HOST is set to localhost.
MediumNEWenhancementAutoBuilderOptionally start shared PRServer with controller 
11211*
11211 Lowering the urgency of this and pushing out to 2.4 as we have mitigated the impact of this by removing the need to re-clone the repository on each oe-selftest run.
MediumNEWmajorAutoBuilderGit clones to NAS are very slow and have significant impact on build times 
11235*
11235

The buildset configuration parser is very unforgiving of syntax errors such as a stray comma[1]. Said stray comma results in a long Traceback which doesn't provide many clues for fixing the problem short of digging into the code, i.e. (from[2]):

Traceback (most recent call last):

 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1187, in unwindGenerator
   return _inlineCallbacks(None, gen, Deferred())
 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1045, in _inlineCallbacks
   result = g.send(result)
 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py", line 169, in upgradeMaster
   master_cfg = loadConfig(config, configFile)
 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py", line 53, in loadConfig
   config['basedir'], configFileName)

--- <exception caught here> ---

 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py", line 149, in loadConfig
   exec f in localDict
 File "/media/data/yocto-autobuilder/yocto-controller/controller.cfg", line 94, in <module>
   yocto_buildsets.createBuildsets()
 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py", line 65, in createBuildsets
   self.parseBuildSet(buildset)
 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py", line 102, in parseBuildSet
   self.parseRepos(buildset)
 File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py", line 127, in parseRepos
   if layer.iterkeys().next() not in self.repos:

exceptions.AttributeError: 'list' object has no attribute 'iterkeys' Errors loading configuration:

 error while parsing config file: 'list' object has no attribute 'iterkeys' (traceback in logfile)

The yocto-autobuilder will be friendlier to administrators if it can help them diagnose such issues more easily by providing more informative reporting of failures, such as pointing to the specific file and line that caused the parser failure.

For extra convenience we could:

  • make the parser more lenient of stray punctuation (like setLenient() in Android's JsonReader[3])
  • switch to using a pure data format, such as JSON, rather than parsing Python AST's. If we were to switch to JSON a user could use their editor of choice's built in linter to detect such issues at editing time.


1. http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/commit/?id=239eeaa1e2cf3a985e75f8b75dedfe704a7d6991 2. https://lists.yoctoproject.org/pipermail/yocto/2017-March/035274.html

3. https://developer.android.com/reference/android/util/JsonReader.html#setLenient(boolean)
MediumNEWenhancementAutoBuilderImprove configuration parser - better error reporting and more lenient 
11251*
11251 EE Peng, according to the backlog list, feature bug 11083 required some kind of automatic validation. I'll reassign this issue to you, so you can confirm it's still valid or close it as invalid/obsolete/wontfix otherwise.
MediumNEWenhancementIoT Reference OS Kit TestingQA: Validate updated components for GPLv3 free Computer Vision profile 
11257*
11257

Buildbot writes observed logs (most commonly stdout and stderr) to the status UI. However, we've noticed that often times we know the build is past a certain step but the logs we can see through the autobuilder status UI haven't been updated to reflect the latest output from bitbake.

We'd like to have the log output synced more frequently, particularly for things like oe-selftest.

It may be that syncing on a timer (i.e. every minute) would be more useful for our needs than syncing on a chunk size (i.e. every 4kb) — assuming that's what currently happens.
MediumNEWenhancementAutoBuilderSync autobuilder log output more frequently 
11261*
11261

We use the wikilog plugin to write autobuilder failures to a page on the Yocto Project wiki, grouped by the triggering build:

https://wiki.yoctoproject.org/wiki/BuildLog

This page is growing very large, which makes it slower to load and intimidating to view for those who want to help manage it.

We should implement log rotation of the wiki page in wikilog so that it only shows either n recent entries, or entries from the past y days.
MediumNEWenhancementAutoBuilderImplement log rotation for Wikilog 
11286*
11286

Hmm, so maybe my understanding of what should and should not work is wrong. I built core-image-minimal for qemux86 off of master commitish 633ad6c9f436f5d2. The following works fine: runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.ext4

The following all fail in diverse ways: uvesa timeout, no login prompt: runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.cpio.gz runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.cpio

No valid filesystem found, kernel panic: runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.tar.bz2 runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic

Undefined video mode 318 (stays stuck for ~ 1 minute then actually booted after I came back from getting tea. I thought his was failing but apparently not...) $ runqemu qemux86 core-image-minimal wic


Does this help clarify?
MediumNEWenhancementOE-Corerunqemu should validate the rootfs type it is trying to use and print clear message if unsupported. 
11346*
11346 Assigning to Cal who will be implementing. He will provide estimate of milestone and person-days.
MediumNEWenhancementDemosCreate demo that shows Amazon Alexa voice activated personal assistant in action 
11347*
11347 Demo should leverage meta-zephyr's ability to perform out of tree builds and multiconfig to build images for each of Arduino 101's three MCU cores. Arduino 101 should communicate over Bluetooth with another device (e.g. Minnowboard Max) also running a Yocto build image. As example Arduino 101 could act as an environmental sensor and Minnowboard as a gateway.
MediumNEWenhancementDemosCreate a demo that shows a Yocto built Zephyr image running on Arduino 101 
11348*
11348 or other custom accelerator would be equally useful/good.
MediumNEWenhancementDemosCreate demo showing Yocto built FPGA image in action 
11358*
11358

Coverage report shows some of the covered lines of wic code as uncovered.

Here is an example:

$ oe-selftest -r wic --coverage $ coverage3 report -m |grep filemap.py /home/ed/git/yocto/poky/scripts/lib/wic/filemap.py 248 99 52 11 55% 79-80, 87-88, 93-94, 102-103, 108-109, 125-126, 138, 147, 160, 169, 183-194, 207-210, 226-249, 253-261, 265, 274-293, 297-299, 303-305, 376, 390-403, 422, 470, 502-516, 530-531, 543, 547-550, 553, 557, 73->76, 78->79, 118->exit, 375->376, 469->470, 542->543, 542->562, 546->547, 546->549, 556->542, 556->557

lines 207-210 belong to FilemapSeek.__init__, which is definitely called in test_sparse_copy test case. The same happens with a lot of other lines of code from this module.

It looks like coverage doesn't work properly for direct API calls. It works only for indirect calls, i.e. when the API is called from the tool/program that's executed in test cases.
MediumNEWnormalAutomated Build Testingoe-selftest: coverage report shows incorrect info 
11364*
11364 I am starting with 5 days for the design phase, and will expand and/or split this task into sub-units as appropriate.
MediumNEWenhancementToasterProvide a wizard-based project creation option 
11371*
11371

1) Add support to point to custom layer index servers.

 * The existing OE Layer Index server should remain the default
 * There should be an easy way to customize the Toaster instance to point to an alternate custom layer index server, for example by adding an entry in "custom.xlm" fixture file.

2) Also add support to point to a layer index summary file, for installations where it is not feasible to gain access or to host a layer index server.

 * The layer index API path should be able to be a file path.
* The format of the file should be in JSON, and should follow the schema of the normal index server content.
MediumNEWenhancementToastersupport custom layer index servers, file-based layer index 
11388*
11388 A build performance visualization tool shows how various features impact build speed.
MediumNEWenhancementDemosCreate a demo using build performance visualization tool to show build speed improvements 
11397*
11397 A bit deeper analysis of buildstats: show the top blocking tasks of bitbake builds. That is, the tasks that are running longest alone, blocking any other task from running.
MediumNEWenhancementOE-Coreoe-build-perf-report: show blocking tasks 
11398*
11398 Make the timeline (i.e. x-axis) of the charts in the html report changeable, e.g. buttons for showing the last 10, 25, 50, 100 and 'all' tested data points.
MediumNEWenhancementOE-Coreoe-build-perf-report: interactive timeline selection in the html report 
11406*
11406

I've occasionally encountered inexplicable build errors and reconfiguration problems. Generally it is from getting a recipe or command incorrect (e.g., wrong URL), attempting to reset the recipe in the workspace and redoing it. The desire is to make the environment pristine for just the recipe desired (or all if -a) without blowing away the entire eSDK (or chasing all the caches to selectively remove), reinstall and rebuild. If it was only a couple locations to remove, I'd have been fine. As I started writing a script to clean the caches and various other locations, I thought it be better if it was included in devtool, itself.

Use cases: Working through steps to refine a demo example workflow for a team. Similarly, reworking steps (and correcting errors in steps) for documentation on examples relevant to a team project. (both variants being where I've encountered these problems).

The very nicety of the cache can come back to haunt you when you don't expect it (e.g., wrong URL, but same filename in URL).

I appreciate not wanting users to shoot themselves in the foot, but there are uses for the -f (even with rm, cp, mv, ln, et al.) The parameter shouldn't be a normal use case (as -f in most cases isn't normal), but one for the exceptional cases.

I would argue that requiring the user to explicitly state they wish to shoot their foot off is acceptable as you can point back to the command issued and state you accepted the terms and consequences thereof.
MediumNEWenhancementOE-CoreAdd --dist-clean, --force, or some other flag to 'devtool reset' to clean all recipe cruft 
11407*
11407

devtool reset [-a|<recipe>] will warn user that the sources/<recipe> will be left behind. Moreover, subsequent issuance of a 'devtool add' will error out on the prior sources/ directory.

However, if the sources/ is removed, and a new 'devtool add' with a corrected URI for the recipe is given, it will use the prior "cached" downloaded version with no failure or warning.

E.g.,

$ devtool add --version 2.4.2 mbedtls https://tls.mbed.org/download/start/mbedtls-2.4.2-apache.tgz

  1. (downloads a landing page, not the desired tarball)

$ devtool reset -a NOTE: Cleaning sysroot for recipe mbedtls... NOTE: Leaving source tree /home/bjkurlex/mbedtls/dev2/workspace/sources/mbedtls as-is; if you no longer need it then please delete it manually

An additional NOTE: here to warn the user that a prior version of the downloaded file (whether valid or not) is left as well.

Otherwise, it is is assumed (even after removing the aforementioned sources/ directory) that a new "corrected" add will work.

  1. issuing "corrected" URI

$ devtool add --version 2.4.2 mbedtls https://tls.mbed.org/download/mbedtls-2.4.2-apache.tgz

This will fail and isn't obvious to a new user that the prior downloaded version is negating the corrected URI even though the sources directory previously warned was removed.

Another approach is to cache the URI associated with the download and if the URI changes (but the file does not), then it should be redownloaded.
MediumNEWnormalOE-Coredevtool should warn user of prior download after reset 
11423*
11423 Is this a superset of bug 11421?
MediumNEWnormalCROPSeSDK automated testcases don't run on Crops-eSDK 
11445*
11445

Here is an example of wks file with a lot of partitions:

$ cat test.wks part ext2 --fstype ext2 --source rootfs part ext3 --fstype ext3 --source rootfs part ext4 --fstype ext4 --source rootfs part squash --fstype squashfs --source rootfs part btrfs --fstype btrfs --size 40M --source rootfs part swap --fstype swap --size 1M

part emptyvfat --fstype vfat --size 1M part emptymsdos --fstype msdos --size 1M part emptyext2 --fstype ext2 --size 1M part emptyext3 --fstype ext3 --size 1M part emptyext4 --fstype ext4 --size 1M part emptybtrfs --fstype btrfs --size 100M

You can generate wic image from this file this way:

$ wic create test.wks -e core-image-minimal INFO: Creating image(s)...

WARNING: bootloader config not specified, using defaults

INFO: The new image(s) can be found here:

 ./test-201705041111-sda.direct

The following build artifacts were used to create the image(s):

 ROOTFS_DIR["ext2"]:           /home/ed/git/yocto/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
 ROOTFS_DIR["ext3"]:           /home/ed/git/yocto/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
 ROOTFS_DIR["ext4"]:           /home/ed/git/yocto/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
 ROOTFS_DIR["squash"]:         /home/ed/git/yocto/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
 ROOTFS_DIR["btrfs"]:          /home/ed/git/yocto/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs
 BOOTIMG_DIR:                  /home/ed/git/yocto/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
 KERNEL_DIR:                   /home/ed/git/yocto/poky/build/tmp/deploy/images/qemux86-64
 NATIVE_SYSROOT:               /home/ed/git/yocto/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot-native

INFO: The image(s) were created using OE kickstart file:

 test.wks


and list partitions this way:

$ /sbin/fdisk -l ./test-201705041111-sda.direct

Disk ./test-201705041111-sda.direct: 233.3 MiB, 244604928 bytes, 477744 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x10118df4

Device Boot Start End Sectors Size Id Type ./test-201705041111-sda.direct1 1 49130 49130 24M 83 Linux ./test-201705041111-sda.direct2 49131 98260 49130 24M 83 Linux ./test-201705041111-sda.direct3 98261 147390 49130 24M 83 Linux ./test-201705041111-sda.direct4 147391 477743 330353 161.3M f W95 Ext'd (LBA) ./test-201705041111-sda.direct5 147392 154151 6760 3.3M 83 Linux ./test-201705041111-sda.direct6 154153 260648 106496 52M 83 Linux ./test-201705041111-sda.direct7 260650 262697 2048 1M 82 Linux swap / Solaris ./test-201705041111-sda.direct8 262699 264746 2048 1M c W95 FAT32 (LBA) ./test-201705041111-sda.direct9 264748 266795 2048 1M 6 FAT16 ./test-201705041111-sda.direct10 266797 268844 2048 1M 83 Linux ./test-201705041111-sda.direct11 268846 270893 2048 1M 83 Linux ./test-201705041111-sda.direct12 270895 272942 2048 1M 83 Linux

./test-201705041111-sda.direct13 272944 477743 204800 100M 83 Linux
MediumNEWenhancementOE-Corewic: allow wic to support additional partition (e.g. 4th partition) 
11458*
11458

The charts in the html report generated by oe-build-perf-report already have "tooltips": a splash window with additional details is shown when hovering mouse cursor over data points. This window could be made much more informative by adding more data, e.g. all the same statistics that are shown for the latest commit on the right side of the chart plus some additional info: - commit number - commit sha1 - number of test runs - mean - +/- - min/max

- standard deviation
MediumNEWenhancementOE-Coreoe-build-perf-report: improve tooltips of html report 
11479*
11479 Investigate whether video analysis on embedded platforms is feasible performance-wise. Integrate relevant software components and see how OpenCV video capabilities can be used with Refkit.
MediumNEWenhancementIoT Reference OS KitComputer vision video analysis support 
11481*
11481 Find out the best method for configuring Refkit images (see email discussion starting from https://lists.yoctoproject.org/pipermail/intel-iot-refkit/2017-March/000007.html). Document the method for Refkit users and do required changes to the framework so that the configuration effort is minimized.
MediumNEWenhancementIoT Reference OS KitRefkit image configuration guidelines 
11487*
11487

The recipe reporting system needs to track meta-intel.

The implementation should cover the posibility to add other layers.
MediumNEWenhancementRecipe Reporting System (RRS)RRS: Add support for meta-intel 
11490*
11490 Evaluate the suitability of existing configuration management solutions (uci/luci, etc.) for integration into refkit.
MediumNEWenhancementIoT Reference OS KitConfiguration Management Solution Study 
11492*
11492 Add documentation on why the profile is needed and how it can be used.
MediumNEWenhancementIoT Reference OS KitReference Kit Gateway Profile Detailed Documentation 
11494*
11494 In order to work with Secure Boot requirement, reduce the need for (L)GPLv3 components. This is done by recompiling components with different configuration parameters, reducing functionality, and leaving components out from the production images
MediumNEWenhancementIoT Reference OS KitGPLv3 free gateway profile 
11523*
11523

The <image>.qemuboot.conf has variables that point to the yocto-autobuilder paths of the image, deploy directory, etc. Since these conf files are in the downloads dir, we should scrub them of the autobuilder specific language so that it is obvious they should be used with the downloadable images and kernels.

Why this matters: The qemuboot.conf for mips64-sato allows it to boot in qemu and have X work. Without the qemuboot.conf, you get a kernel panic from the wrong machine memory. If you force the correct memory size, you still get a blank X11 screen...

If someone naively looks at the conf file, they are unlikely to use it for the downloaded images due to the many autobuilder paths in it.
MediumNEWnormalOE-Coreqemuboot.conf files in the downloads.../machines/qemu/qemu<machine> should be scrubbed of ab paths 
11530*
11530

Thinking about this again, shim will probably do this out of the box.

It might be just a case of verifying if this is the case.
MediumNEWenhancementBSPsRMC:[EFI] modify shim to launch an EFI binary when not running in secure boot mode 
11531*
11531 When running in non-secure boot context, RMC should be able to add the fingerprint of the current board to the data store. The fingerprint generation function will be shared between EFI and userspace contexts.
MediumNEWenhancementBSPsRMC: [EFI] Add register board with data store functionality (non-secure boot) 
11532*
11532

Parse CONFIG file and store RMC.efi log to file if

LOG=/path/to/log is found.
MediumNEWenhancementBSPsRMC: [EFI] Add log option 
11533*
11533 Parse CONFIG file and enable verbosity if DEBUG=1 is found.
MediumNEWenhancementBSPsRMC: [EFI] Add debug option 
11534*
11534

Parse the tasks file and process the tasks defined in it. The only supported task at present is CP (copy a file from the datastore to a target location).

Example TASKS file:

FP=123456789ABCDEFG

  CP SRC_FILE TARGET_FILE


If the board fingerprint is 123456789ABCDEFG execute task "CP SRC_FILE TARGET_FILE".
MediumNEWenhancementBSPsRMC: [EFI] Process tasks in TASKS file 
11535*
11535

When running in secure boot mode, RMC.efi has to verify the integrity of the data store. This involves generating and MD5 hash over the contents of the data store and comparing it to the MD5 has stored in the RMC.efi PE header (last field in data directories).

https://en.wikipedia.org/wiki/Portable_Executable#/media/File:Portable_Executable_32_bit_Structure_in_SVG_fixed.svg

If the system is running in secure boot mode and the two hashes do not match the boot process should be terminated with an appropriate message to the user.
MediumNEWenhancementBSPsRMC: [EFI] Verify data store integrity (secure boot) 
11538*
11538

This should be something like (-b for bless):

./rmc -b -d data_store_dir -f RMC.efi
MediumNEWenhancementBSPsRMC: Bless data store 
11539*
11539

Verity that the MD5 hash stored in the RMC.efi PE header matches the MD5 hash generated over contents of the data store.

Example:

./rmc -v -d data_store_dir -f RMC.efi
MediumNEWenhancementBSPsRMC: Verify RMC.efi is properly blessed 
11540*
11540

Print the contents of an existing fingerprint file - both the SMBIOS strings stored in the file and the MD5 hash corresponding to these strings.

Example:

./rmc -p -f FILE.fp
MediumNEWenhancementBSPsRMC: Print fingerprint file contents 
11543*
11543 RMC should be able to add the fingerprint of the current board to the data store. The fingerprint generation function will be shared between EFI and userspace contexts.
MediumNEWenhancementBSPsRMC: Register board with data store 
11544*
11544 Parse CONFIG file and enable verbosity if DEBUG=1 is found.
MediumNEWenhancementBSPsRMC: Add debug option 
11545*
11545

Parse CONFIG file and store RMC log to file if

LOG=/path/to/log is found.
MediumNEWenhancementBSPsRMC: Add log option 
11546*
11546

It would be useful to have configurable thresholds for measurement results. Both absolute and relative thresholds should be supported. If the result (mean) would exceed the threshold it would be noted in the report. HTML report coloring would probably be adjusted so that only results exceeding the threshold would be colored red/green.

Exceeding a threshold would also be reflected in the return value of the script. This way, the script could be used in bisecting possible regressions.
MediumNEWenhancementOE-Coreoe-build-perf-report: implement thresholds for measurement results 
11548*
11548 This is important work but we're unlikely to block milestone releases on it so I've set the target to just 2.4 in the expectation that it will be a medium priority defect. This makes sense since upstream developers actions are hard to schedule.
MediumNEWenhancementOE-CoreReduce local Pending patches 
11596*
11596

bzip2-ptest requires Makefile. This file contains numerous references to the host build system (see the attached diffoscope output). These are harmless and also unneeded, as the ptest runs this:

$ make -k runtest

and the "runtest" target in the Makefile is:

runtest: ./bzip2 -1 < sample1.ref > sample1.rb2 ./bzip2 -2 < sample2.ref > sample2.rb2 ./bzip2 -3 < sample3.ref > sample3.rb2 ./bzip2 -d < sample1.bz2 > sample1.tst ./bzip2 -d < sample2.bz2 > sample2.tst ./bzip2 -ds < sample3.bz2 > sample3.tst @if cmp sample1.bz2 sample1.rb2; then echo "PASS: sample1 compress";\ else echo "FAIL: sample1 compress"; fi @if cmp sample2.bz2 sample2.rb2; then echo "PASS: sample2 compress";\ else echo "FAIL: sample2 compress"; fi @if cmp sample3.bz2 sample3.rb2; then echo "PASS: sample3 compress";\ else echo "FAIL: sample3 compress"; fi @if cmp sample1.tst sample1.ref; then echo "PASS: sample1 decompress";\ else echo "FAIL: sample1 decompress"; fi @if cmp sample2.tst sample2.ref; then echo "PASS: sample2 decompress";\ else echo "FAIL: sample2 decompress"; fi @if cmp sample3.tst sample3.ref; then echo "PASS: sample3 decompress";\ else echo "FAIL: sample3 decompress"; fi


The references to the host build system prevent the bzip2-ptest package from being binary reproducible.

One option is to sanitize all host references, the other (probably better) is to create/install only the relevant part of the Makefile
MediumNEWenhancementOE-Corebzip2-ptest: improve reproducible build 
11601*
11601 There are cases where series contain patches (.patch) but these are not referenced in the corresponding recipe so a (patchtest) check is is needed.
MediumNEWenhancementPatchwork/Patchtestcheck new patches are indicated in the SRC_URI 
11602*
11602

The image with acl-ptest contains this file:/usr/lib/acl/ptest/include/builddefs

The file references the host build path on several occasion (see the attachment), hence the file ("builddefs") should be sanitized. This is needed not only for binary reproducibility, but to actually pass the test.
MediumNEWnormalOE-Coreacl-ptest references host build path 
11629*
11629 Add a T&T article for a Yocto Poky image that can be booted on a de10-nano on the HPS arm SOC. This is so we can detail how to build and run a Poky image in addition to the default Angstrom one that is the current default.
MediumNEWenhancementDemosdocument in tips and tricks how to bring up a de10-nano altera HPS with Yocto Poky 
11656*
11656

libtool contains two instances of host build references:

1. LTCFLAGS:

LTCFLAGS="·​-​O2·​-​pipe·​-​g·​-​feliminate-​unused-​debug-​types·​-​fdebug-​prefix-​map=/​tmp/​ramdisk/​build-​repro-​no-​dpkg-​2017-​06-​12/​work/​i586-​poky-​linux/​libtool/​2.​4.​6-​r0=/​usr/​src/​debug/​libtool/​2.​4.​6-​r0·​-​fdebug-​prefix-​map=-​native=·​-​fdebug-​prefix-​map==·​-​ffile-​prefix-​map=/​tmp/​ramdisk/​build-​repro-​no-​dpkg-​2017-​06-​12/​work/​i586-​poky-​linux/​libtool/​2.​4.​6-​r0/​libtool-​2.​4.​6=/​libtool-​2.​4.​6/​·​·​-​ffile-​prefix-​map=/​tmp/​ramdisk/​build-​repro-​no-​dpkg-​2017-​06-​12/​work/​i586-​poky-​linux/​libtool/​2.​4.​6-​r0/​libtool-​2.​4.​6=/​libtool-​2.​4.​6/​

2: lt_truncate_bin="/​tmp/​ramdisk/​build-​repro-​no-​dpkg-​2017-​06-​12/​hosttools/​dd·​bs=4096·​count=1"

These not only prevent binary reproducibility, but I would assume prevent proper functionality as well. I believe the remedy would be to set them as:

LTCFLAGS="·​-​O2·​-​pipe·​-​g·​-​feliminate-​unused-​debug-​types

lt_truncate_bin="dd·​bs=4096·​count=1"
MediumNEWnormalOE-Corelibtool contains build host references 
11663*
11663 I bet this is because you're using arch...
MediumNEWnormalOE-CoreBuild QA RPATH warning in perl-native build (HiRes.so) 
11666*
11666 /usr/lib/attr/ptest/include/builddefs contains various references to the host build system. This breaks binary reproducibility but possibly the ptest as well.
MediumNEWnormalOE-Coreattr-ptest references host build path 
11667*
11667

usr/lib/flex/ptest/Makefile needs to be sanitized.

Please see the attached file.
MediumNEWnormalOE-Coreflex-ptest references host build path 
11668*
11668

./usr/lib/zlib/ptest/Makefile contains some references to the build host system that should be removed.

Please see the attached file.
MediumNEWnormalOE-Corelibz-ptest contains build host references 
11674*
11674

"The cpio utility was standardized in POSIX.1-1988, but it was omitted from POSIX.1-2001 because of its file size (and other) limitations. For example, the GNU version offers various output format options, such as "bin" (default, and obsolete) and "ustar", having a file size limitations of 2,147,483,647 bytes (2 GB) and 8,589,934,591 bytes (8 GB), respectively.[3]"

Very much looks like file size is the problem. If we're not using ustar format we should, and if possible we should verify the size.
MediumNEWnormalOE-Corecore-image-sato-sdk-ptest build error for cpio image 
11681*
11681

This looks like a 48 core system and I suspect its running too many things in parallel and running out of resources. Please try something like BB_NUMBER_THREADS = "20" and PARALLEL_MAKE = "-j 20" and see if the issue still occurs.

Monitoring the number of open files and active processes (etc.) during a running build would help narrow down which kinds of resource the system is running out of.
MediumNEWmajorBSPspoky-tiny fails to build because of resource unavailability 
11704*
11704 adding markus and randy
MediumNEWenhancementBitBakeAdd other resource monitoring options to conf/local.conf STOPTASKS/ABORT 
10224*
10224 Not sure about the terminal codec , considering the reproducibility is very low , I am to OK to close this as 'Won't fix'.
MediumRESOLVEDnormalAutomated Runtime Testingoeqa/utils/sshcontrol.py:SSHProcess decode stdout data fail 
11027*
11027

Change pushed to production in commit:

0a48d51f84f3ab14a93cb932b39e1235789c4ae3
MediumRESOLVEDenhancementPatchwork/Patchtestpatchwork: Notify if status-change-through-email was attempted without required privileges 
11200*
11200

Change pushed to production in commit:

0866a65ab53102f14c5fd3e7da4c23ade3bf1d7d
MediumRESOLVEDnormalPatchwork/Patchtestpatchwork: Bundle paging links drop "archive" query parameter 
11229*
11229

This should not really be fixed in RMC. RMC uses whatever processor type is reported by smbios to generate a board fingerprint. Qemu modifies the processor type field in each of its releases. Therefore to support multiple qemu releases with the same rmc.db record, we should use a common machine type by passing "-machine" parameter when qemu is started.

https://lists.yoctoproject.org/pipermail/meta-intel/2017-March/004702.html
MediumRESOLVEDnormalBSPsDifferent major versions of QEMU produce different RMC fingerprints 
11244*
11244 Closed in agreement with the previous comments from Jaska. The feature already has two test cases to validate the pulseaudio modules are loaded, currently located on https://github.com/intel/intel-iot-refkit/blob/master/meta-iotqa/lib/oeqa/runtime/multimedia/audio/pulseaudio.py
MediumRESOLVEDenhancementIoT Reference OS Kit TestingQA: Validate Reference kit has BT audio support on gateway profile 
11246*
11246

Closed in agreement with the previous comments from Patrick. The image installer feature already has three test cases to validate basic installation scenarios. Tests are currently located on https://github.com/intel/intel-iot-refkit/blob/master/meta-refkit-core/lib/oeqa/selftest/image-installer.py

As stated by Patrick, QA still needs to determine the best way to track the development of additional tests that may be added later.
MediumRESOLVEDenhancementIoT Reference OS Kit TestingQA: Validate Reference kit supports image installer 
11294*
11294

The requested functionality is provided in commit:

f7c2ec9f8719ee37bdca866c881ca5d853f9aefc
MediumRESOLVEDenhancementOE-Corecreate-pull-request: add in-reply-to option to ensure patch/series threads don't break 
11320*
11320 Won't apply to RMC2.
MediumRESOLVEDenhancementBSPsRMC: Extend grub-efi to process RMC database file from ESP 
11321*
11321 Does not apply to RMC2.
MediumRESOLVEDenhancementBSPsRMC: grub-efi - append KBOOTPARAM to linux boot entry's cmdline 
11338*
11338 Won't apply to RMC2. Data store will be a directory tree.
MediumRESOLVEDenhancementBSPsRMC: records cannot be added to and removed from an existing RMC database 
11405*
11405 Does not apply to RMC2.
MediumRESOLVEDnormalBSPsRMC: records in an existing RMC database cannot be updated 
11446*
11446 This requirement is not valid now.
MediumRESOLVEDenhancementBSPsFlexibility and configurability to boot from any partition 
11456*
11456

I'm fine with following the sensible Debian policy. Thanks for the explanation.

The problem we encountered was due to an upper case LINUX_VERSION_EXTENSION as Joe explained.
MediumRESOLVEDnormalOE-CoreLINUX_VERSION_EXTENSION cannot contain capital letters for ipk packaging 
11462*
11462 The PR #200 got merged.
MediumRESOLVEDenhancementIoT Reference OS KitROS-Core support 
11465*
11465 Got merged to intel-iot-refkit
MediumRESOLVEDenhancementIoT Reference OS KitReference Kit Industrial Robotics Profile Detailed Documentation 
11476*
11476 Replace iptables with nftables. Add new features provided by nftables, like modular file-based firewall configuration. Support multiple network interfaces classified to different logical networks (LAN, WAN, VPN, ...). Make performance comparisons between the firewalls.
MediumRESOLVEDenhancementIoT Reference OS KitNftables firewall support 
11477*MediumRESOLVEDenhancementIoT Reference OS KitDetailed computer vision profile documentation 
11485*
11485 Now nodejs version updated to 6.11 in refkit .
MediumRESOLVEDenhancementIoT Reference OS KitRefkit: Upgrade nodejs version(v5.11+) 
11541*
11541

This bug covers the implementation of the MD5 fingerprint generator, which is now complete. The implementation of the different methods for reading smbios data is in progress and is covered by this bug:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=11537
MediumRESOLVEDnormalBSPsRMC: print the fingerprint and MD5 hash of the current board 
11542*MediumRESOLVEDnormalOE-Corerunqemu does not display the error text from qemu-system-XXX when qemu-system-XXX fails to run 
11588*
11588 PR #167 is merged.
MediumRESOLVEDnormalIoT Reference OS KitCan't resolve own hostname when running under QEMU 
11699*
11699 Thanks - Setting status to RESOLVED and putting the doc flag to "done".
MediumRESOLVEDnormalDevelopment Manual[megamanual] dnf using rpm: 5.18.4.3.1. Using RPM section changes required 
11158*
11158 Currently there isn't any automated testing for multiconfig. I’ll expand the scope of the change using bug 11612 to track it and make the globbing testing part of the overall multiconfig tests. I'll also update the milestone to account for this.
Medium+ACCEPTEDnormalAutomated Build Testingbitbake: Create test for multiconfig globing support 
11202*
11202

Hi Ee Peng,

If you decide no more tests will be added for this milestone, I would suggest waiting for the patchset to trickle down from OpenEmbedded first as mentioned on bug 11086. Once the patchset is in the repository and bug 11086 is closed, you could close this too by updating it to "Resolved" as you indicate.
Medium+ACCEPTEDenhancementIoT Reference OS Kit TestingQA: Validate Flatpak-based 3rd party application support 
11394*
11394 Bugs 11390 and 11391 demonstrate that there's a bit of a gap in our testing of devtool and git URLs. We should have some tests that verify the conditions that trigger those bugs once they are fixed, as well as tests that exercise the behaviour implemented in reformat_git_uri() in scripts/lib/recipetool/create.py.
Medium+ACCEPTEDenhancementAutomated Runtime TestingAdd tests to cover devtool handling of various git URL styles 
11424*
11424

Add Link Time Optimization (LTO) to poky-tiny's kernel.

Some work has already been submitted and its been merged on yocto-kernel-cache and linux-yocto MLs, although this work was for kernel 4.4.

A rebase to this previous work needs to be done to make it work with newer production kernels, the reason why this is not merged for newer kernels is that there are some compilation/linking issues with kernels > 4.4.
Medium+ACCEPTEDenhancementMeta-yoctopoky-tiny + Link Time Optimization 
11454*
11454

We would like to rationalise the buildset configuration we use on the public autobuilder to better support the project moving forwards and to remove technical debt and legacy items which hinder interactions with the AB.

A non-complete list of desired changes:

  • change targets and ordering of targets
  • possibly split oe-selftest run up
  • remove nightly-uclibc and other legacy cruft
Medium+ACCEPTEDenhancementAutoBuilderRationalise autobuilder configuration 
11455*
11455

We'd like to rationalise the buildset configuration used by the YP clusters (see bug #11454) but in order to do so whilst maintaining support for building older releases of the Yocto Project we need to be able to easily switch between an autobuilder instance running with the new configuration and an autobuilder instance running with the legacy configuration.

It should be possible to implement a simple script to manage two copies of the yocto-autobuilder (new config and legacy config) and switch between the two cleanly from clusters controller.

This could also enable updating of the new config's buildbot and friends to make use of newer upstream code.
Medium+ACCEPTEDenhancementAutoBuilderImplement script to support running two versions of AB on YP infrastructure 
11466*
11466

Currently, whole-disk encryption only works with TPM v1.2. TPM in firmware (aka fTPM) is based on TPM v2.0.

This issue here covers: - enabling the TPM 2.0 userspace = https://github.com/01org/tpm2.0-tools - adding the necessary kernel configuration (if needed) - adding a "tpm2.0" image and distro feature, with the corresponding changes to

 the code which currently only supports tpm1.2 (initramfs, installer): as for 
 1.2, usage of 2.0 is meant to be optional with parameters to make it mandatory

- testing under qemu, using the experimental

 https://github.com/stefanberger/swtpm/tree/tpm2-preview
  • Not* covered is testing on real hardware. Joule might work, but testing and automating that is something that needs to be done by QA.
Medium+ACCEPTEDenhancementIoT Reference OS KitTPM 2.0 support (under qemu) 
11468*
11468

The implementation will be based on UEFI-based capsule update capabilities in the firmware.

There are two projects that we should consider for integration: https://github.com/rhinstaller/fwupdate http://fwupd.org.s3-website-eu-west-1.amazonaws.com/

This issue is about: - providing recipes for either fwupdate or fwupd - support for firmware updates as part of the system update (bug #11467), i.e.

 getting new firmware onto a device and triggering its installation, including
 a reboot
Testing whether the actual firmware gets installed by fwupdate or fwupd is out of scope of this issue, because it is hardware-specific.
Medium+ACCEPTEDenhancementIoT Reference OS Kitintegrate firmware update into system update 
11497*
11497

Some buildsteps to run this test have now been enabled on the Yocto Autobuilder:

http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/commit/?id=5d80b30d55c134aeca63643b4ef3611a1845c6d1

However this isn't complete because the way to ensure that shared state objects are used it to generate a locked signature file and use that when running the build. However, as reported in bug #11623, when auto.conf includes a relative path to the generated locked-sigs.inc this value is propagated to the simulated eSDK environment where bitbake is run during eSDK generation, but the file isn't and its absence causes a build error like:

ERROR: core-image-sato-1.0-r0 do_populate_sdk_ext: Failed to generate filtered task list for extensible SDK:

      1. Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:

   core-image-minimal
   core-image-sato
   meta-toolchain
   meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86' ERROR: bitbake failed: ERROR: ParseError at /srv/autobuilder/yocto-autobuilder/yocto-worker/nightly-qa-extras/build/build/newtmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/sdk-ext/image/tmp-renamed-sdk/conf/local.conf:23: Could not include required file ../locked-sigs.inc ERROR: core-image-sato-1.0-r0 do_populate_sdk_ext: Function failed: copy_buildsystem ERROR: Logfile of failure stored in: /srv/autobuilder/yocto-autobuilder/yocto-worker/nightly-qa-extras/build/build/newtmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/temp/log.do_populate_sdk_ext.49644 NOTE: recipe core-image-sato-1.0-r0: task do_populate_sdk_ext: Failed ERROR: Task (/srv/autobuilder/yocto-autobuilder/yocto-worker/nightly-qa-extras/build/meta/recipes-sato/images/core-image-sato.bb:do_populate_sdk_ext) failed with exit code '1'

NOTE: Tasks Summary: Attempted 7344 tasks of which 6862 didn't need to be rerun and 1 failed.
Medium+ACCEPTEDnormaleSDKeSDK: create test case for populate_sdk_ext using shared states 
11587*
11587

(In reply to comment #4) > Since Hongxu has submitted this patch to gcc and opened a gcc bug [1] as > well, the flag -ffile-prefix-map=from-string=to-string *could* show up in > the version of gcc used on the host. It's not clear what will be implemented > [2]. > > An explicit flag that is only present if the compiler was configured as a > cross-compiler would be better. > > Playing devil's advocate, I'll point out that this only catches the common > case of a single toolchain being installed on the host. I don't think that > we can or should make this impervious to arbitrary build host configurations > with, for example, additional cross-compilers available. > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268 > [2] https://reproducible-builds.org/specs/build-path-prefix-map/

Yes, there is a non-zero possibility the host compiler supports the flag. This can be tested by querying the gcc:

$ gcc -Q --help=joined | grep ffile-prefix-map

I suspect the patch, if accepted, will be not backported and will end up in gcc 7.x . My point was, that in order to achieve binary reproducibility (which was an implied objective here), they should have used the flag anyway. If the host gcc dies in the process, even better.

But I will come up with a gcc-cross specific/unique flag patch as well.
Medium+ACCEPTEDenhancementOE-Coreadd "fence flag" to cross toolchain 
11680*
11680

(In reply to comment #1) > Is this effectively an enhancement request to ensure .wic images are > generated even on release (i.e. krogoth) where the system doesn't generate > them by default?

For "krogoth" is not necessary to publish .wic images as after 2.1.3 this branch will not have QA cycles, but for "morty" and onwards .wic images will be needed even for point releases.
Medium+ACCEPTEDenhancementAutoBuilderEnable AB to build WIC images for qemux86 and qemux86-64 machines and publish them 
11176*
11176

Add optional post-process command to use strip-nondeterminism to remove sources of non-derterminism from the rootfs.

This will provide an optional "hammer" to improve the determinism of the system's output whilst we continue to work on the wider problem of reproducible builds with the Yocto Project.

Post-processing a rootfs with strip-nondeterminism will provide immediate benefits to users of the system performing file-based updates (i.e. we'll be better able to produce a new rootfs for processing by OSTree which only includes real changes, not just changes introduced by non-determinism).
Medium+IN PROGRESS DESIGNenhancementOE-CoreAdd optional command to rootfs-postprocess to remove non-determinism from rootfs 
11177*
11177

Pass options to tar, et al, to write archives with deterministic metadata (clamp uid & gid to 0, stable timestamp, etc).

Some archive formats don't provide this functionality OOTB, for those we should consider other techniques to ensure the archives are deterministic across builds.

https://reproducible-builds.org/docs/archives/
Medium+IN PROGRESS DESIGNenhancementOE-CoreGenerate archives with deterministic metadata 
11178*
11178

Probably worth mentioning: in local.conf:

PACKAGE_CLASSES = "package_rpm package_deb package_ipk"

BUILD_REPRODUCIBLE_BINARIES="1"
Medium+IN PROGRESS DESIGNenhancementOE-CoreUse SOURCE_DATE_EPOCH 
11179*
11179

Locale and Timezone settings of a shell in which builds are performed can affect the build output in various ways[1][2], adding non-determinism to the system when building on machines with different TZ settings.

We should export a consistent TZ and locale to the environment in which builds are performed in order to improve the determinism of our build output.

1. https://reproducible-builds.org/docs/timezones/

2. https://reproducible-builds.org/docs/locales/
Medium+IN PROGRESS DESIGNenhancementOE-CoreEnsure TZ is also set to a consistent value 
10880*
10880

I had asked Richard about this one via email, but there are other higher priority items for 2.3 to complete first.

I'll take my half done implementation and complete it in early 2.4
Medium+IN PROGRESS IMPLEMENTATIONnormalOE-Coreperf recipe contaminates linux shared workdir in do_configure_prepend() 
11308*
11308 Tried bitbake world for the pkgconf but it seems to show this error. I've attached the error log on this to provide you with more detail on this issue.
Medium+IN PROGRESS IMPLEMENTATIONenhancementOE-CoreConsider replacing pkg-config with pkgconf 
11429*
11429 perhaps by pointing to s oe-selftest config that makes it explicit which variables we are excluding and including?
Medium+IN PROGRESS IMPLEMENTATIONenhancementOE-Coreoe-selftest should use its own build directory and write the desired configuration 
11434*
11434

right now devtool edit-recipe has ugly tracebacks if executed w/o an EDITOR available. This happens in the build containers. It would be nice to 1) say  : no EDITOR defined/available. exiting politely....

2) provide a devtool find-recipe which prints out the path to the recipe.
Medium+IN PROGRESS IMPLEMENTATIONenhancementOE-Coredevtool edit-recipe/find-recipe enhancements 
11447*
11447 I sent an RFC patchset to meta-intel which adds refkit's combo app. The combo app is capable of booting the kernel directly and should satisfy this need.
Medium+IN PROGRESS IMPLEMENTATIONenhancementBSPsConfigurability to boot from UEFI boot partition or direct boot without bootloader 
11451*
11451

(In reply to comment #7) > (In reply to comment #6) > > Two issues still arises. > > 1. I can build using devshell manually, however, running over bitbake juci > > still fails stating that it cannot find "minify" despite minify existing > > inside the recipe-sysroot-native and "minify" can be used during devshell. > > So I'm still investigating why this is happening. > > I don't know anything about the Juci build machinery but it's > possible/probably that when building it manually within the devshell it's > using various utilities from your host. > > If you haven't already, I recommend the devshell section of the YP manual: > http://www.yoctoproject.org/docs/2.3/mega-manual/mega-manual.html#platdev- > appdev-devshell

Thanks, it appears that minify had some issues due to permissions problem. After looking thru, I found out that inside node_modules there exist a .bin directory that is created by "npm install" each time it is called. The issue is that all of the softlinks inside the .bin directory had wrong permissions. I didn't spend too much time into investigating why this is happening yet, I'll probably re-visit and optimize this better once I've completed resolving other more critical issues inside Juci. Currently I'm prioritizing integrating Juci into Poky.

The investigation on why the Juci build dir is empty is still on-going and I have some pre-early reports which is not conclusive yet but I'll require some effort and time to attempt this and find more clues.

Looking thru the documentation on Juci, it looks like there's more steps to go thru to integrate Juci that the recipe provided by meta-openwrt might not have covered. Then again, this is just some early reporting and this is merely a guess based on Juci docs and recipes from meta-openwrt. Looking thru the Juci documentation, it looks like they are tightly bound to OpenWRT's method of building, i.e. using feeds and defconfig to configure the builds of Juci. (see here https://github.com/mkschreder/juci-openwrt-feed)

Since I'm not sure if they are building it correctly, my proposal to understand how they build it would be to attempt building an OpenWRT with Juci on an actual router and try to compare between their environment and their builds. The purpose is to find any gaps in the current recipe provided inside meta-openwrt and check if it is building correctly. And if required, modification on the recipe maybe needed in order to make this work. So my next step is now to replicate OpenWRT with Juci on my own device (LinkIt SMART 7688) and see how it goes. Hopefully I'm not going about the long way :/ I'll probably increase the Person-Days to 17 since the amount of effort seems to have increased.
Medium+IN PROGRESS IMPLEMENTATIONenhancementOE-CoreBring in Juci from openWRT as an on target network configurator 
11628*
11628 one of the goals for this implementation is to be able to use the bootloader to write to flash so that we do not need proprietary tools to create and program the flash images.
Medium+IN PROGRESS IMPLEMENTATIONenhancementOE-Coreimplement uboot for nios2 softcore on max10 reference board 
10368*
10368 Moving the target to 2.4 M2 as the patch was not merged on 2.4 M1.
Medium+IN PROGRESS REVIEWenhancementBitBakeQA Bitbake: create setup and basic unit tests for event module 
11316*
11316

Patchset posted for review (only patches 2 to 5 are relevant):

http://lists.openembedded.org/pipermail/openembedded-core/2017-May/136269.html
Medium+IN PROGRESS REVIEWenhancementOE-CoreTrim down LSB support 
11610*Medium+IN PROGRESS REVIEWenhancementOE-Coregdk-pixbuf: improve reproducibility 
11687*
11687 When I say "remove them" I mean remove mountprog=21111,nfsprog=11111, from the nfsroot line.
Medium+IN PROGRESS REVIEWnormalGeneral RuntimeCan not run QEMU machines under UNFS 
11717*
11717

The solution is to break the query into parts.

diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py index 2efb0fd5..39014379 100755 --- a/lib/toaster/toastergui/views.py +++ b/lib/toaster/toastergui/views.py @@ -483,10 +483,15 @@ def builddashboard( request, build_id ):

        npkg = 0
        pkgsz = 0
        package = None

- for package in Package.objects.filter(id__in = [x.package_id for x in t.target_installed_package_set.all()]): - pkgsz = pkgsz + package.size - if package.installed_name: - npkg = npkg + 1 + # Chunk the query to avoid "too many SQL variables" error + package_set = t.target_installed_package_set.all() + package_set_len = len(package_set) + for ps_start in range(0,package_set_len,500): + ps_stop = min(ps_start+500,package_set_len) + for package in Package.objects.filter(id__in = [x.package_id for x in package_set[ps_start:ps_stop]]): + pkgsz = pkgsz + package.size + if package.installed_name: + npkg = npkg + 1

        elem['npkg'] = npkg
elem['pkgsz'] = pkgsz
Medium+IN PROGRESS REVIEWnormalToasterlarge package set causes 'too many SQL variables' error 
10541Medium+NEWenhancementCROPSUse "new project template" framework to create sample Devtool project 
10543Medium+NEWenhancementCROPSuse CDT "new build system" to enable building for Make project 
10544Medium+NEWenhancementCROPSuse CDT "new build system" to enable building for CMake project 
10545Medium+NEWenhancementCROPSuse CDT "new build system" to enable building for Automake project 
10546*
10546 probably future... but we'll see.
Medium+NEWenhancementCROPSuse CDT "new build system" to enable building for Devtool project 
10548Medium+NEWenhancementCROPSEclipse CROPS plugin support for debugging via gdb to QEMU 
10549Medium+NEWenhancementCROPSEclipse CROPS plugin support for debugging via gdb to hardware target 
10556Medium+NEWenhancementCROPSDocker tooling support to manipulate CROPS containers from the Eclipse CROPS plugin 
11131*
11131

There are some tests that needs to be made based on YP Compatible registration form [1]:

- Test case to review if the README file specifies the maintainer. - Test case to review if the README file specifies where to send patches. - Test case to review if the README file contains the dependencies listed, this needs some format definition to be auditable?.

- Test case to build what the layer provides, at this time only parsing is made.

[1] https://www.yoctoproject.org/webform/yocto-project-compatible-registration
Medium+NEWenhancementMeta-yoctoAdd more tests to Yocto Project Compatible Layers script 
11183*
11183

We've run into several issues trying to integrate cve-check-tool into the YP workflow and the upstream is less active than we'd like.

We could write our own Python tool to integrate more cleanly with YP. Instead of harvesting the full CVE database we could simplify both the tool and the work it generates by scraping LWN's curated weekly CVE list?
Medium+NEWenhancementOE-Core"native" CVE scanning solution 
11209*
11209

Okay, I added a workaround, so the latest dnf/libdnf should work now. You can base the feed signing work on top of this branch:

https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=akanavin/package-version-updates
Medium+NEWnormalOE-CoreRPM package feed signing is currently not supported 
11360*
11360

Currently, the report always draws one line per chart in the html report (representing one tester/branch/machine). It would be nice to be able to draw multiple lines per chart, i.e. for example represent test results from multiple testers in the same chart, for example.

This will probably require a new report template. For example, the statistics currently shown are not necessarily meaningful when multiple data sets are represented in one chart.
Medium+NEWenhancementOE-Coreoe-build-perf-report: show multiple data sets per chart 
11361*
11361 Utilize the system monitoring feature of buildstats that samples the system state (cpu/mem/disk utilization) at regular intervals. The sampled data shall be collected and stored along with other buildstats.
Medium+NEWenhancementAutomated Build Testingoe-build-perf-test: monitor system resource utilization 
11381*
11381

Make it possible to display a summary of bitbake tasks. This information is extracted from buildstats and is only available for tests/measurements where buildstats are collected.

We could display e.g. the following information: - total number of tasks - top 5 resource-hungry tasks (cputime)

- top 5 change in resource usage (cputime)
Medium+NEWenhancementOE-Coreoe-build-perf-report: show task summary 
11382*
11382

We could display the changes in build content for measurements for which buildstats are saved: - new recipes - removed recipes

- recipe version changes
Medium+NEWenhancementOE-Coreoe-build-perf-report: display changes in build content 
11383*
11383

At the moment you can run an individual task and its dependencies within a recipe (as per bitbake -b) but you can't build a target as normal with all dependencies honoured. We could use this immediately for example in order to avoid having to shut down tinfoil when we need to build nodejs-native or kern-tools-native.

(Some care will need to be taken in order to ensure there's no contamination between executions if we run more than one in succession, and that the events fired make sense.)
Medium+NEWenhancementBitBaketinfoil: enable running tasks with dependencies 
11457*
11457

Not that long ago we added a prefix to logged messages to allow us to see where the message has been generated (recipe / task). This is undoubtedly useful for errors and warnings, however, I'm not sure it's really appropriate for bb.plain(). As an example, see the output for -c listtasks now:

... NOTE: Executing RunQueue Tasks nodejs-native-4.5.0-r0 do_listtasks: do_addto_recipe_sysroot nodejs-native-4.5.0-r0 do_listtasks: do_build Default task for a recipe - depends on all other normal tasks required to 'build' a recipe nodejs-native-4.5.0-r0 do_listtasks: do_checklicense nodejs-native-4.5.0-r0 do_listtasks: do_checklicenseall nodejs-native-4.5.0-r0 do_listtasks: do_checkpkg ...

This is really ugly. Can we exclude bb.plain from having these prefixes added?
Medium+NEWminorBitBakebb.plain() isn't plain anymore 
11483*
11483

Automate first 7 testcases from testplan

https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=207
Medium+NEWenhancementAutomated Runtime TestingAutomate testcases for package management 
11488*
11488

Currently we don't have tests for ipk/opkg when generate image with package_ipk.

The tests needs to cover,

opkg,

- Sanity execute opkg -h. - Install/Remove some package.

apt,

- Sanity execute opkg -h. - Configure from remote package feed. - Search package from package feed.

- Install package from package feed.
Medium+NEWenhancementAutomated Runtime TestingTestimage: Add tests for ipk/opkg like dnf/rpm ones 
11517*
11517 there is a check for the sign-off-by tag, but this enhancement is about the name itself, i.e. failing in case of single names, etc.
Medium+NEWenhancementPatchwork/Patchtestcheck for real sign-off-by names 
11518*
11518

According to [1], the submitted and inappropriate upstream-states must provide info to either where the patch was submitted or why the patch is inappropriate.

[1] http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines#Patch_Header_Recommendations
Medium+NEWenhancementPatchwork/PatchtestUpstream-Status Submitted and Inappropriate check for where/reason info 
11519*
11519

(In reply to comment #1) > (In reply to comment #0) > > According to [1] "Python functions must be four space indented - no tabs." > > > > [1] http://www.openembedded.org/wiki/Styleguide#Format_Guidelines > > There's a Pylint option that catches tab indentation. We are already using > Pylint to test Python code and I think changing a couple parameters will do > the work. Do you think a new check is needed?

understood, then the pylint implementation should be the path to take.
Medium+NEWenhancementPatchwork/Patchtestcheck for four-space indented in python metadata 
11521*
11521

This occurs if you use the downloaded mips image and downloaded mips kernel, but have no qemuboot.conf.

you get an early kernel panic and a "Data bus error" right after Enabling cache parity protection.
Medium+NEWnormalOE-Corerunqemu defaults to 512M memory if QB_MEM is not set, breaking mips64 
11522*
11522

having env variables to control aspects of qemu settings should accept the value and the runqemu script should handle the necessary command line syntactic sugar. This would seem to be more intuitive if all you do is read the variable and try to set it.


This should be done with all the env variables that runqemu uses that currently expect to have full qemu command lines set.
Medium+NEWenhancementOE-Corerunqemu env var QB_MEM expects to be set to "-m 256" not "256" or "256M" 
11547*
11547

Comments from RP:

About ptest, we don't have good tools to show "what regressed" at the moment and we don't trigger ptest as part of any of our autobuilder builds. There therefore could be some work in this area. Enabling it as part of our image testing, collecting up the results and showing where things regress in some way.
Medium+NEWenhancementOE-CoreAdd ptest to testimage 
11609*
11609

We should ensure builds on the latest Ubuntu release are sound, to that end we need to have an Ubuntu 17.04 builder on our cluster.

Let's wait for QA to test that everything is OK with that host distro (bug #11608) before proceeding.
Medium+NEWenhancementAutoBuilderAdd an Ubuntu 17.04 builder to the YP AB cluster 
11627*
11627 max10m50c board boots with 3.18, we need to update the nios2 softcore on the board to boot with the 4.10 kernel.
Medium+NEWenhancementOE-Coreupdate max10 reference board to boot with 4.10 kernel 
11642*
11642 Work on GDC DAFT instance to support all the DUT's used on BSP QA cycles and execute automated tests.
Medium+NEWenhancementAutomated Runtime TestingUse DAFT to execute BSP's automated test on QA cycles. 
11679*
11679

More information, I also tested with the following scenarios: ... MACHINE = "genericx86-64" TCLIBC = "musl" DISTRO_FEATURES_append = " ld-is-gold" ...

... MACHINE = "beaglebone" TCLIBC = "glibc" DISTRO_FEATURES_append = " ld-is-gold" ...

... MACHINE = "genericx86-64" TCLIBC = "glibc" DISTRO_FEATURES_append = " ld-is-gold" ...

bitbake core-image-minimal

the issue did not show up with any of the above configs, so it only impacts arm + bfd linker?
Medium+NEWmajorOE-Core"cannot make copy relocation for protected symbol" build errors with musl and ld-is-gold 
11700*
11700

Robert, can you take this. tcl makes me so crazy. :)

The paths in the files are now pointing into the recipe sysroot so all the paths are breaking. Note that there's a patch on the list against this recipe to clean up some of the low hanging fruit I found before deciding I didn't want to learn enough to fix this confidently.
Medium+NEWnormalOE-Coretcl_8.6.6: tclConfig.sh: non-deterministic generation 
11702*
11702

If local.conf doesn't have a newline at the end, we get this error:

$ oe-selftest -r archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name 2017-06-22 11:40:18,363 - oe-selftest - INFO - Adding layer libraries: 2017-06-22 11:40:18,363 - oe-selftest - INFO - /media/build1/poky/meta/lib 2017-06-22 11:40:18,364 - oe-selftest - INFO - /media/build1/poky/meta-yocto-bsp/lib 2017-06-22 11:40:18,364 - oe-selftest - INFO - /media/build1/poky/meta-selftest/lib 2017-06-22 11:40:18,364 - oe-selftest - INFO - Running bitbake -p 2017-06-22 11:40:23,385 - oe-selftest - INFO - 2017-06-22 11:40:23,386 - oe-selftest - INFO - Running tests... 2017-06-22 11:40:23,498 - oe-selftest - INFO - ---------------------------------------------------------------------- 2017-06-22 11:40:46,005 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver) ... ERROR (22.506s) 2017-06-22 11:40:46,006 - oe-selftest - INFO - 2017-06-22 11:40:46,006 - oe-selftest - INFO - ====================================================================== 2017-06-22 11:40:46,006 - oe-selftest - INFO - ERROR [22.506s]: test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver) 2017-06-22 11:40:46,006 - oe-selftest - INFO - ---------------------------------------------------------------------- 2017-06-22 11:40:46,006 - oe-selftest - INFO - Traceback (most recent call last):

 File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
   return func(*args, **kwargs)
 File "/media/build1/poky/meta/lib/oeqa/selftest/cases/archiver.py", line 33, in test_archiver_allows_to_filter_on_recipe_name
   src_path = os.path.join(bb_vars['DEPLOY_DIR_SRC'], bb_vars['TARGET_SYS'])
 File "/usr/lib/python3.5/posixpath.py", line 89, in join
   genericpath._check_arg_types('join', a, *p)
 File "/usr/lib/python3.5/genericpath.py", line 143, in _check_arg_types
   (funcname, s.__class__.__name__)) from None

TypeError: join() argument must be str or bytes, not 'NoneType' 2017-06-22 11:40:46,007 - oe-selftest - INFO - ---------------------------------------------------------------------- 2017-06-22 11:40:46,007 - oe-selftest - INFO - Ran 1 test in 22.507s 2017-06-22 11:40:46,007 - oe-selftest - INFO - 2017-06-22 11:40:46,007 - oe-selftest - INFO - FAILED (errors=1) 2017-06-22 11:40:46,007 - oe-selftest - INFO - 2017-06-22 11:40:46,007 - oe-selftest - INFO - Generating XML reports... 2017-06-22 11:40:46,011 - oe-selftest - INFO - RESULTS: 2017-06-22 11:40:46,011 - oe-selftest - INFO - RESULTS - archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name - Testcase 1345: ERROR 2017-06-22 11:40:46,011 - oe-selftest - INFO - SUMMARY: 2017-06-22 11:40:46,011 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 22.626s 2017-06-22 11:40:46,011 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed

We can likely do better than this.
Medium+NEWnormalAutomated Build Testingoe-selftest fails if local.conf doesn't end with a newline 
10942*Medium+RESOLVEDnormalOE-Corelinux-firmware-ath9k fw ver 1.4.0 in main package 
10999*
10999

Merged the libva change to oe-core:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=73d1c2713725602549fff99434be24fe593d80bb

and into meta-intel here for libva-intel-driver:

http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/commit/?id=0c2da994195b4db36db80258d09d0e5faed22136

Both now use the github repo directly and when updates occur we will need to update the SRCREV SHAs
Medium+RESOLVEDmajorBSPsmeta-intel: Recipes multimedia:libva need to be updated to point to github upstream projects 
11002*
11002 Merged in oe-core e9471f8da3946439141ccdd8284200aa614df46c.
Medium+RESOLVEDnormalOE-Coredbus: user-session dbus.socket contains path from native sysroot 
11164*Medium+RESOLVEDnormalMeta-yoctoyocto-compat-layer: Add support to isolate the environment 
11180*
11180

We landed DNF within the Yocto 2.3 timeframe as part of our effort to move the userspace to Python3 by default. Unfortunately there wasn't time to complete this effort and the RPM4 & DNF we currently build are using Python2.

We should move all of these components to Python3 in the 2.4 cycle.
Medium+RESOLVEDenhancementOE-CoreSwitch RPM4, dnf and co to Python3 
11181*
11181

Patchset posted for review: http://lists.openembedded.org/pipermail/openembedded-core/2017-April/136138.html

From the cover letter, this is the sitation now:

There are still a few py2 holdouts: - python-numpy (needed by a few recipes in meta-oe, shares code with python3-numpy) - python-nose (dependency of python-numpy) - python-scons (work to port it to py3 is ongoing now) - python-setuptools (dependency of python-scons) - bmap-tools - ltp - perf - asciidoc

And a couple of packagegroups list python 2 as well: - packagegroup-self-hosted - packagegroup-core-lsb

Let's revisit the situation with the above in a year or so.
Medium+RESOLVEDenhancementOE-CoreRemove Python 2 as a host tool or needing to be built for our core images 
11182*
11182

(2) is solved by oe-core 54a2549802a911cad2475a6aa379315a834419d8.

I now think that (1) isn't required, as people won't be accidentally including python3-core in images anymore.
Medium+RESOLVEDenhancementOE-CoreMake it easier to add Python to an image 
11295*
11295 All 3 patches have been merged into master.
Medium+RESOLVEDnormalOE-Coredo_image_wic: fails when rm_work is active 
11305*
11305

This fix is deployed in production:

9b3b3b49188ee7954b653f295a66498d4b8ac88a
Medium+RESOLVEDenhancementPatchwork/Patchtestpatchwork: Improve Series naming when no cover letter is provided 
11313*
11313 Fixed in oe-core 767335c92b7cc657a008722a908380a3c89c3c66.
Medium+RESOLVEDnormalOE-CoreAs a developer, I want to avoid churn of cross,crosssdk,nativesdk,cross-canadian do_populate_sysroot 
11323*
11323 Does not apply to RMC2. Fingerprints will be directory names and name collisions will not be allowed.
Medium+RESOLVEDmajorBSPsRMC: Handle fingerprints and file blob name collisions in rmc.bbclass 
11355*
11355

Merged in oe-core master in: http://git.openembedded.org/openembedded-core/commit/?id=e06266798d975bd6bebdb6bfdbd3d21be1c44ffd http://git.openembedded.org/openembedded-core/commit/?id=496a9dc179fe9dc370c940f4a2f7bcab869a804f

It is now possible to dump buildstats from the build perf results Git repository and analyze them with the buildstats-diff script.
Medium+RESOLVEDenhancementOE-Coreoe-build-perf-report: support viewing buildstats 
11414*Medium+RESOLVEDnormalOE-Coredevtool breaks if an invalid image name is passed to build-image 
11428*
11428

A new buildset, nightly-refkit, has been added (as well as supporting new code and buildsteps). nightly-refkit has two main builds:

1) clones the intel-iot-refkit repository and all submodules then sets up a bblayers.conf which refers to the OE-Core and Bitbake repositories in the main poky checkout with layers from the refkit repository. The buildset then builds whatever the value of REFKIT_CI_BUILD_TARGETS is in meta-refkit/conf/distro/include/refkit-ci.inc as well as running oe-selftests.

2) builds the same REFKIT_CI_BUILD_TARGETS from meta-refkit/conf/distro/include/refkit-ci.inc with the poky distro

http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/commit/?id=1c4e352100d457d99f7096401e8834bc75d29141
Medium+RESOLVEDenhancementAutoBuilderSupport building intel-iot-refkit with yocto-autobuilder 
11574*
11574 Fixed in oe-core ad915e9baa04c73981c4795a97da95cea40b50c2.
Medium+RESOLVEDnormalOE-CoreERROR: kconfig-frontends-4.10.0.1-r0 do_compile: Function failed: 
11575*
11575

Patch merged into OE-Core and backports for older releases:-

OE-Core rev: dc8b21ae0ed3bceb9f3df4f6cd8f8f55b9c306fb

Poky master rev: cd1addaaf4f4681465de6b0ca705e01a5058e3ba

Poky pyro rev: 31389f8b0088a8a83aa5e50f11359b39934e4640

Poky morty rev: e92165f5cea1c345672dd866df6a44d1cd8b97ce

Poky krogoth rev: 80b35ed1a282a4f3f5d723febfd70481a48182c2
Medium+RESOLVEDnormalOE-CoreERROR: cryptodev-tests-1.8-r0 do_checkuri: Fetcher failure for URL: 'http://download.gna.org/cryptodev-linux/cryptodev-linux-1.8.tar.gz' 
11608*
11608 Ubuntu17.04 has been added on GDC/AB as supported distro
Medium+RESOLVEDnormalAutomated Build Testingdistro-testing: Add ubuntu 17.04 to supported distros 
11635*
11635 So is this due to build machine lacking swap as discussed in 11608? I guess I can close this as NOTABUG.
Medium+RESOLVEDnormalOE-Core32bit x86 webkitgtk builds fail on Ubuntu 17.04 
11636*
11636 Closing as not a bug: the test machine had less swap than the current autobuilder instances and run out of memory. Both piglit and especially webkit have very high peak memory usage: 32 GB may not be enough for a world build if large values of PARALLEL_MAKE are used.
Medium+RESOLVEDnormalOE-Core32bit x86 piglit builds fail on Ubuntu 17.04 host 
11682*
11682 This case is covered by other bug.
Medium+RESOLVEDnormalAutomated Build Testingoe-selftest does not clean-up properly the build/conf metadata 
11705*
11705

Looking at the generated wrappers for gobject-introspection, it shows several references to the host build path (and build user name!). That cannot be right.

Please see the attached file.
UndecidedNEWnormalOE-Coregobject-introspection_1.50.0 contains various references to the host build system 
11706*
11706

the way oe-setup-rpmrepo searches for createrepo_c does not work with RSS. It should probably just use/leverage oe-run-native.

Docs: we need to document how to use the repo from the target side as well.
UndecidedNEWnormalOE-Coreoe-setup-rpmrepo no longer works with RSS 
11708*
11708

When python3 unittest-xml-reporting is installed (i.e. "import xmlrunner" works), then oe-selftest produces a directory called "TestResults_<date>_<number>" with individual .xml files.

We should enable that in our CI system by installing unittest-xml-reporting for Python3 and publish the resulting .xml files so that Jenkins shows the same way as DAFT test results.

unittest-xml-reporting wasn't packaged for Debian, but installation with "sudo pip3 install unittest-xml-reporting" worked fine.
UndecidedNEWnormalIoT Reference OS KitCI: capture and display results of oe-selftest as JUnit XML 
11709*
11709

Currently, the WIC and HDDIMG images produced by yocto fails to boot when written to a storage device that presents itself as having 4096 byte logical (and physical) sectors.

Changing the FAT cluster size from 512 to 4096 in meta/classes/image-live.bbclass allows a valid file system on such a device.

4K sectors/cluster breaks legacy BIOS boot, so an additional MACHINE_FEATURES_remove = "pcbios" is required in local.conf.

Make a new MACHINE_FEATURES entry for 4K cluster EFI system partitions?
UndecidedNEWmajorOE-CoreSupport EFI System partition with 4K FAT clusters 
11710*
11710

Unfortunately http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=415fbfb0cd1bc5129179596ba27ae2362b7de2a4 has caused a regression - it can't work with memory resident bitbake, since it attempts to call functions defined in the metadata on the UI side instead of in the bitbake server with the following result:


snip ------------

$ recipetool create -V 2.4.2 -N mbedtls https://tls.mbed.org/download/start/mbedtls-2.4.2-apache.tgz NOTE: Fetching https://tls.mbed.org/download/start/mbedtls-2.4.2-apache.tgz... ERROR: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module>

    0001:
*** 0002:extend_recipe_sysroot(d)
    0003:

File: '/home/paul/poky/poky/meta/classes/staging.bbclass', lineno: 350, function: extend_recipe_sysroot

    0346:    # Detect bitbake -b usage
    0347:    nodeps = d.getVar("BB_LIMITEDDEPS") or False
    0348:    if nodeps:
    0349:        lock = bb.utils.lockfile(recipesysroot + "/sysroot.lock")
*** 0350:        staging_populate_sysroot_dir(recipesysroot, recipesysrootnative, True, d)
    0351:        staging_populate_sysroot_dir(recipesysroot, recipesysrootnative, False, d)
    0352:        bb.utils.unlockfile(lock)
    0353:        return
    0354:

Exception: NameError: name 'staging_populate_sysroot_dir' is not defined ...


snip ------------ It could be that we should fix this at the same time as bug 11198 as executing this as more traditional do_fetch + do_unpack tasks would bypass the problem.
UndecidedNEWnormalOE-CoreFetching from recipetool no longer works with memres 
11711*
11711 The shortlog's prefix indicates several things besides indicating that it is a patch: revision, targeted branch, patch number and total patches. This info needs to be present in a standard format, thus making it easier to parse it. Create a tests to check for the latter.
UndecidedNEWnormalPatchwork/Patchtestcheck proper prefix on shortlog 
11713*
11713 We *could* patch iptables but a simpler solution would be to change iptables to something else.
UndecidedNEWnormalAutomated Runtime Testingtestimage for core-image-sato-sdk fails on musl due to iptables compile failures 
11714*
11714

STATUS: FAILED BUILD: 2017-06-22_02-28-09-build-212 ENVIRONMENT: intel-corei7-64 - on MinnowBoard Turbot 64bit EXPANSION BOARD: Silverjaw Lure (SKU 1000) WIRELESS MODULE: Intel Dual Band Wireless-AC 7260 - 7260HMW STEPS TO REPRODUCE:

   With Bluetooth already enabled, execute the following commands:
   connmanctl disable bluetooth
   connmanctl enable bluetooth

EXPECTED OUTCOME:

   Bluetooth is enabled and the hci0 interface responds to HCI commands.

ACTUAL:

   After disabling Bluetooth, the next error appears on the Kernel buffer:
   [ 421.158820] Bluetooth: hci0: turning off Intel device LED failed (-61)
   Following the error, the hci0 interface stops responding and any attempt to bring the interface up times out until the DUT is rebooted.

ADDITIONAL NOTES:

   Reloading the btusb module throws the following output to the kernel buffer:
   [  502.420138] usbcore: deregistering interface driver btusb
   [  511.156381] usbcore: registered new interface driver btusb
   [  513.165857] Bluetooth: hci0 command 0x0c03 tx timeout
   [  521.615945] Bluetooth: hci0 sending initial HCI reset command failed (-110)
   This issue also occurs on the Meta-Intel build 2.3_M3_meta_intel_rc1.
Similar occurrence on Red Hat: https://bugzilla.redhat.com/show_bug.cgi?id=1290011
UndecidedNEWnormalIoT Reference OS KitIntel AC 7260 Bluetooth interface stops responding after disabling Bluetooth 
11010*
11010

Great - setting to RESOLVED and marking the doc flag to "done."

Thanks!
UndecidedRESOLVEDnormalReferencedocumentation for NO_RECOMMENDATIONS is incorrect 
11489*
11489 duplicate of Bug 11460
UndecidedRESOLVEDnormalOE-Corepostinst-useradd uses the abs path of pseudo which is encoded in sstate 
11514*
11514

I couldn't reproduce the issue in same system, same configuration the very next day.

I'm closing this bug as invalid now.
UndecidedRESOLVEDnormalOE-Corem4_native: do_configure failure on Centos-7 
11573*
11573

Bug in a patch, now dropped from mut.

Marking as OBSOLETE so that QA don't want to verify.
UndecidedRESOLVEDnormalOE-CoreERROR: lzop-1.03-r0 do_package_write_ipk 
11576*
11576 The elfutils patch was merged to meta-gpl2 master today.
UndecidedRESOLVEDnormalOE-CoreERROR: elfutils-0.148-r11 do_compile: oe_runmake failed 
11577*
11577 Oops, filed the same bug twice.
UndecidedRESOLVEDnormalOE-CoreERROR: elfutils-0.148-r11 do_compile: oe_runmake failed 
11617*
11617 The patches have been removed from MUT.
UndecidedRESOLVEDnormalOE-Core[wic] test_wic_rm failed 
11619*
11619

$ git clone git://anonscm.debian.org/collab-maint/ca-certificates.git Cloning into 'ca-certificates'... remote: Counting objects: 1678, done. remote: Compressing objects: 100% (994/994), done. remote: Total 1678 (delta 1178), reused 925 (delta 664) Receiving objects: 100% (1678/1678), 915.56 KiB | 465.00 KiB/s, done. Resolving deltas: 100% (1178/1178), done.

Transient failure on the cluster.
UndecidedRESOLVEDnormalOther YP Layersca-certificates build fails due to invalud URL 
11624*
11624 MUT-only, patch has been dropped and the submitter told.
UndecidedRESOLVEDnormalOE-Corecore-image-sato fails dnf testimage run 
11625*
11625 Transient failure.
UndecidedRESOLVEDnormalOther YP Layersdropbear fails checkuri 
11626*
11626 This was user error and the subsquent build was ok, sorry for the noise.
UndecidedRESOLVEDnormalBitBakeFailed to get distroversion. DISTRO_VERSION="2.1.3" is not the same as specified Yocto Project release number 
11641*
11641 Already yanked from MUT.
UndecidedRESOLVEDnormalOE-Coretest_integrity_compressed_images oe-selftest failed 
11678*
11678 This seems to be a duplicate. Please, have a look at the bug 11525 and add your requirements there. Thanks.
UndecidedRESOLVEDnormalOE-CoreFeature Request: /etc/fstab to use PARTUUID for EFI wic images 
Personal tools