Weekly Status: Difference between revisions

From Yocto Project
Jump to navigationJump to search
Line 1: Line 1:
== Yocto Project Weekly Status May 18, 2021 ==
== Yocto Project Weekly Status May 25, 2021 ==
Current Dev Position: YP 3.4 M1<br/>
Current Dev Position: YP 3.4 M1<br/>
Next Deadline: 7th June 2021 YP 3.4 M1 build<br/>
Next Deadline: 7th June 2021 YP 3.4 M1 build<br/>


'''Next Team Meetings:'''<br/>
'''Next Team Meetings:'''<br/>
*Bug Triage meeting Thursday May 20th at 7:30am PDT (https://zoom.us/j/454367603?pwd=ZGxoa2ZXL3FkM3Y0bFd5aVpHVVZ6dz09)
*Bug Triage meeting Thursday May 27th at 7:30am PDT (https://zoom.us/j/454367603?pwd=ZGxoa2ZXL3FkM3Y0bFd5aVpHVVZ6dz09)
*Monthly Project Meeting Tuesday June 1st at 8am  PDT (https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09)
*Monthly Project Meeting Tuesday June 1st at 8am  PDT (https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09)
*Weekly Engineering Sync Tuesday May 18th at 8am PDT (https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09)
*Weekly Engineering Sync Tuesday June 8th at 8am PDT (https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09)
*Twitch -  See https://www.twitch.tv/theyoctojester
*Twitch -  See https://www.twitch.tv/theyoctojester


'''Key Status/Updates:'''<br/>
'''Key Status/Updates:'''<br/>
*YP 3.2.4 was released
*YP 3.3.1 was released
*YP 3.3.1 is in QA
*YP 3.1.8 is being built, an autobuilder glitch meant we’re building rc2.
*Patch merging to master is continuing and we now have a round of AUH upgrades to process. We’ve tried to merge some of the easier ones as there was autobuilder testing availability. This allows people to concentrate on the areas where there are issues with the automated upgrade and where we most need the help.
*We’ve seen a significant reduction in CVE counts for master and the stable releases this week. For master this brings us down to 5 CVEs compared to 63 a couple of weeks ago. This should mean real/open issues are much clearer and old/obsolete issues no longer contribute noise to the output of the tools.
*Anuj is unavailable for a few weeks so Richard will be acting as maintainer for hardknott until he returns (assistance welcome).
*Help with the problematic recipe upgrades would be much appreciated.
*An open letter about open source project work was sent to various mailing lists highlighting some of the challenges established project face from a resourcing perspective: https://lists.openembedded.org/g/openembedded-architecture/topic/open_source_maintainers_an/82722442
*The multiconfig changes in bitbake continue to cause problems, we still need simpler test cases to reproduce issues rather than huge builds. The existing patches seem to fix some workloads and break others. Richard is trying to fix but trying to fix autobuilder issues and other problems and these are slow builds to debug.
*The multiconfig changes in bitbake continue to cause problems, we still need simpler test cases to reproduce issues rather than huge builds. The existing patches seem to fix some workloads and break others. Richard is trying to fix but trying to fix autobuilder issues and other problems and these are slow builds to debug.
*We saw a significant drop in open CVEs reported by the tooling against master this week (63 down to 18) and hopefully many more of these have resolutions “in flight”. This will allow us to focus on the real issues and filter out the legacy noise.
*We have identified ltp issues with some tests causing hangs which have been disabled as well as issues with the 5.10.37 kernel which seem to be fixed in 5.10.38.
*SMP was enabled for qemu on arm/x86 and we switched x86 to a more modern cpu/platform
*We’re seeing odd looking rcu stalls in qemu automated testing. This seems to occur more often with 5.10 kernel but also with 5.12 and with kvm-clock disabled and we have seen it with 5.4. It is unclear whether these are kernel issues or qemu issues, we’re struggling to debug them and any insight would be welcome.
*One source of the OOM issues on the autobuilder appears to be glibc usermode testing going out of control and using up all system memory. We are investigating the best way to mitigate that but it accounts for some of our intermittent issues.
*Some debugging of these intermittent hangs has been posted to the swat mailing list.
*When OOM issues occur an issue was identified in bitbake’s heartbeat events causing bitbake to hang. Ironically this was being heavily used by the recent autobuilder load debugging code. There is a patch pending to address this.
*A rare sstate manifest corruption bug was tracked down thanks to information from Martin Jansa, a fix has been merged.
*We have enabled more resource control on the autobuilder for xz memory/thread usage as that was contributing to the OOM issues. More investigation is needed into the rpm/deb controls for xz, help there would be appreciated.
*Intermittent autobuilder issues continue to occur and are now at a record high level. You  can see the list of failures we’re continuing to see by searching for the “AB-INT” tag in bugzilla: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=AB-INT We are working to identify the load pattern on the infrastructure that seems to trigger these.
*Intermittent autobuilder issues continue to occur and are now at a record high level. You  can see the list of failures we’re continuing to see by searching for the “AB-INT” tag in bugzilla: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=AB-INT We are working to identify the load pattern on the infrastructure that seems to trigger these.


Line 40: Line 36:


'''Planned upcoming dot releases:'''<br/>
'''Planned upcoming dot releases:'''<br/>
*YP 3.2.4 is in released
*YP 3.3.1 is in released
*YP 3.3.1 is in QA
*YP 3.1.8 is being built
*YP 3.3.1 release date 2021/05/28
*YP 3.1.8 build date 2021/05/24
*YP 3.1.8 release date 2021/06/04
*YP 3.1.8 release date 2021/06/04
*YP 3.1.9 build date 2021/06/21
*YP 3.1.9 build date 2021/06/21
Line 55: Line 49:


'''Tracking Metrics:'''<br/>
'''Tracking Metrics:'''<br/>
*WDD 2664 (last week 2674) (https://wiki.yoctoproject.org/charts/combo.html)
*WDD 2647 (last week 2664) (https://wiki.yoctoproject.org/charts/combo.html)
*Poky Patch Metrics   
*Poky Patch Metrics   
**Total patches found: 1297 (last week 1331)
**Total patches found: 1282 (last week 1297)
**Patches in the Pending State: 494 (38%) [last week 497 (37%)]
**Patches in the Pending State: 491 (38%) [last week 494 (38%)]


'''The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:'''<br/>
'''The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:'''<br/>

Revision as of 14:40, 25 May 2021

Yocto Project Weekly Status May 25, 2021

Current Dev Position: YP 3.4 M1
Next Deadline: 7th June 2021 YP 3.4 M1 build

Next Team Meetings:

Key Status/Updates:

  • YP 3.3.1 was released
  • YP 3.1.8 is being built, an autobuilder glitch meant we’re building rc2.
  • We’ve seen a significant reduction in CVE counts for master and the stable releases this week. For master this brings us down to 5 CVEs compared to 63 a couple of weeks ago. This should mean real/open issues are much clearer and old/obsolete issues no longer contribute noise to the output of the tools.
  • Help with the problematic recipe upgrades would be much appreciated.
  • The multiconfig changes in bitbake continue to cause problems, we still need simpler test cases to reproduce issues rather than huge builds. The existing patches seem to fix some workloads and break others. Richard is trying to fix but trying to fix autobuilder issues and other problems and these are slow builds to debug.
  • We have identified ltp issues with some tests causing hangs which have been disabled as well as issues with the 5.10.37 kernel which seem to be fixed in 5.10.38.
  • We’re seeing odd looking rcu stalls in qemu automated testing. This seems to occur more often with 5.10 kernel but also with 5.12 and with kvm-clock disabled and we have seen it with 5.4. It is unclear whether these are kernel issues or qemu issues, we’re struggling to debug them and any insight would be welcome.
  • Some debugging of these intermittent hangs has been posted to the swat mailing list.
  • Intermittent autobuilder issues continue to occur and are now at a record high level. You can see the list of failures we’re continuing to see by searching for the “AB-INT” tag in bugzilla: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=AB-INT We are working to identify the load pattern on the infrastructure that seems to trigger these.

Ways to contribute:

YP 3.4 Milestone Dates:

  • YP 3.4 M1 build date 2021/06/07
  • YP 3.4 M1 Release date 2021/06/18
  • YP 3.4 M2 build date 2021/07/12
  • YP 3.4 M2 Release date 2021/07/23
  • YP 3.4 M3 build date 2021/08/23
  • YP 3.4 M3 Release date 2021/09/03
  • YP 3.4 M4 build date 2021/10/04
  • YP 3.4 M4 Release date 2021/10/29

Planned upcoming dot releases:

  • YP 3.3.1 is in released
  • YP 3.1.8 is being built
  • YP 3.1.8 release date 2021/06/04
  • YP 3.1.9 build date 2021/06/21
  • YP 3.1.9 release date 2021/07/02
  • YP 3.3.2 build date 2021/07/19
  • YP 3.3.2 release date 2021/07/30
  • YP 3.1.10 build date 2021/07/26
  • YP 3.1.10 release date 2021/08/06
  • YP 3.1.11 build date 2021/09/13
  • YP 3.1.11 release date 2021/9/24

Tracking Metrics:

The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:
https://wiki.yoctoproject.org/wiki/TSC

Archives

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Weekly_Status&oldid=84717"