Weekly Status: Difference between revisions

From Yocto Project
Jump to navigationJump to search
 
(308 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Yocto Project Weekly Status May 18, 2021 ==
== Yocto Project Weekly Status June 4th, 2024 ==
Current Dev Position: YP 3.4 M1<br/>
Current Dev Position: YP 5.1 M1 <br/>
Next Deadline: 7th June 2021 YP 3.4 M1 build<br/>
Next Deadline: YP 5.1 M1 Build 20 May 2024<br/>


'''Next Team Meetings:'''<br/>
'''Next Team Meetings:'''<br/>
*Bug Triage meeting Thursday May 20th at 7:30am PDT (https://zoom.us/j/454367603?pwd=ZGxoa2ZXL3FkM3Y0bFd5aVpHVVZ6dz09)
*Bug Triage meeting Thursday June 6th at 7:30 am PDT (https://zoom.us/j/454367603?pwd=ZGxoa2ZXL3FkM3Y0bFd5aVpHVVZ6dz09)
*Monthly Project Meeting Tuesday June 1st at 8am  PDT (https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09)
*Weekly Engineering Sync Tuesday June 4th at 8 am PDT (https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09)
*Weekly Engineering Sync Tuesday May 18th at 8am PDT (https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09)
*Twitch -  See https://www.twitch.tv/theyoctojester
*Twitch -  See https://www.twitch.tv/theyoctojester


'''Key Status/Updates:'''<br/>
'''Key Status/Updates:'''<br/>
*YP 3.2.4 was released
*YP 4.0.19 is in QA
*YP 3.3.1 is in QA
*YP 5.1 M1 rc1 is building
*Patch merging to master is continuing and we now have a round of AUH upgrades to process. We’ve tried to merge some of the easier ones as there was autobuilder testing availability. This allows people to concentrate on the areas where there are issues with the automated upgrade and where we most need the help.
*We identified the performance problem in bitbake using hash equivalence servers and have made changes to batch queries and improve build times
*Anuj is unavailable for a few weeks so Richard will be acting as maintainer for hardknott until he returns (assistance welcome).
*Extra logging for that performance problem is available by setting BB_LOGCONFIG=XXX/bitbake/contrib/autobuilderlog.json (the more verbose logging the autobuilder uses) which will then show if initial query or rehash looping takes longer than 30s for iterations or 60s overall.
*An open letter about open source project work was sent to various mailing lists highlighting some of the challenges established project face from a resourcing perspective:
*We changed locked-sigs to include the hash equivalence mapping data, removing the need to copy the cache file.
https://lists.openembedded.org/g/openembedded-architecture/topic/open_source_maintainers_an/82722442
*Libtool was upgraded to the 2.5.0 alpha release, many patches were merged upstream.
*The multiconfig changes in bitbake continue to cause problems, we still need simpler test cases to reproduce issues rather than huge builds. The existing patches seem to fix some workloads and break others. Richard is trying to fix but trying to fix autobuilder issues and other problems and these are slow builds to debug.
*Qemu has been upgraded to 9.0.0, kernel headers to 6.9 and stable and dev kernel updates merged (to 6.6.32)
*We saw a significant drop in open CVEs reported by the tooling against master this week (63 down to 18) and hopefully many more of these have resolutions “in flight”. This will allow us to focus on the real issues and filter out the legacy noise.
*There is one remaining issue on fedora40, systemtap-native does not build. The other Fedora 40 are believed to be fixed.
*SMP was enabled for qemu on arm/x86 and we switched x86 to a more modern cpu/platform
*Some layers are setting fallbacks for UNPACKDIR which break OE-Core by trying to claim compatibility with everything. Errors have been added to make this problem clear.
*One source of the OOM issues on the autobuilder appears to be glibc usermode testing going out of control and using up all system memory. We are investigating the best way to mitigate that but it accounts for some of our intermittent issues.
*sstate has had changes to fix a bug where build-history data was inconsistent. The siteconfig class was removed to make that change easier and improve the code, it was mostly unused anyway.
*When OOM issues occur an issue was identified in bitbake’s heartbeat events causing bitbake to hang. Ironically this was being heavily used by the recent autobuilder load debugging code. There is a patch pending to address this.
*The parallelism in the hash client was removed as batching queries is much more effective and easier on the server.
*A rare sstate manifest corruption bug was tracked down thanks to information from Martin Jansa, a fix has been merged.
*There is an open letter the project has created related to the CVE/NVD situation, more information is available here: https://lists.openembedded.org/g/openembedded-architecture/message/1990
*We have enabled more resource control on the autobuilder for xz memory/thread usage as that was contributing to the OOM issues. More investigation is needed into the rpm/deb controls for xz, help there would be appreciated.
*Please consider signing this to show support for those changes.
*Intermittent autobuilder issues continue to occur and are now at a record high level. You  can see the list of failures we’re continuing to see by searching for the “AB-INT” tag in bugzilla: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=AB-INT We are working to identify the load pattern on the infrastructure that seems to trigger these.
*Signatures from small businesses/consultancies are as welcome as those from larger organizations as we want to demonstrate the breadth of the need for these changes.
*There is a further update from Marta/Syslinbit on the CVE tooling/patches
*We continue to have trouble tracking down the rust reproducibility problem in rustdoc.
*Post release upgrading is resulting in an improvement in patch metrics which is great to see, more should appear next week in the statistics too.


'''Ways to contribute:'''<br/>
'''Ways to contribute:'''<br/>
*As people are likely aware, the project has a number of components which are either unmaintained, or have people with little to no time trying to keep them alive. These components include: devtool, toaster, wic, oeqa, autobuilder, CROPs containers, pseudo and more. Many have open bugs. Help is welcome in trying to better look after these components!
*There are bugs identified as possible for newcomers to the project: https://wiki.yoctoproject.org/wiki/Newcomers
*There are bugs identified as possible for newcomers to the project: https://wiki.yoctoproject.org/wiki/Newcomers
*There are bugs that are currently unassigned for YP 3.3. See: https://wiki.yoctoproject.org/wiki/Bug_Triage#Medium.2B_3.3_Unassigned_Enhancements.2FBugs
*There are bugs that are currently unassigned for YP 5.1. See: https://wiki.yoctoproject.org/wiki/Bug_Triage#Medium+_5.1_Unassigned_Enhancements/Bugs
*We’d welcome new maintainers for recipes in OE-Core. Please see the list at: http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/conf/distro/include/maintainers.inc and discuss with the existing maintainer, or ask on the OE-Core mailing list. We will likely move a chunk of these to “Unassigned” soon to help facilitate this.
*We’d welcome new maintainers for recipes in OE-Core. Please see the list at: http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/conf/distro/include/maintainers.inc and discuss with the existing maintainer, or ask on the OE-Core mailing list. We will likely move a chunk of these to “Unassigned” soon to help facilitate this.
*Help is very much welcome in trying to resolve our autobuilder intermittent issues. You can see the list of failures we’re continuing to see by searching for the “AB-INT” tag in bugzilla: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=AB-INT.
*Help us resolve CVE issues: CVE metrics
*We have a growing number of bugs in bugzilla, any help with them is appreciated.


'''YP 3.4 Milestone Dates:'''<br/>
'''Tracking Metrics:'''<br/>
*YP 3.4 M1 build date 2021/06/07
*WDD 2772 (last week 2772) (https://wiki.yoctoproject.org/charts/combo.html)
*YP 3.4 M1 Release date 2021/06/18
*OE-Core/Poky Patch Metrics
*YP 3.4 M2 build date 2021/07/12
**Total patches found: 1098 (last week 1121)
*YP 3.4 M2 Release date 2021/07/23
**Patches in the Pending State: 205 (19%) [last week 221 (20%)]
*YP 3.4 M3 build date 2021/08/23
*https://autobuilder.yocto.io/pub/non-release/patchmetrics/
*YP 3.4 M3 Release date 2021/09/03
*YP 3.4 M4 build date 2021/10/04
*YP 3.4 M4 Release date 2021/10/29


'''Planned upcoming dot releases:'''<br/>
'''YP 5.1 Milestone Dates:'''<br/>
*YP 3.2.4 is in released
*YP 5.1 M1 Build date 2024-05-20
*YP 3.3.1 is in QA
*YP 5.1 M1 Release date 2024-05-31
*YP 3.3.1 release date 2021/05/28
*YP 5.1 M2 Build date 2024-07-08
*YP 3.1.8 build date 2021/05/24
*YP 5.1 M2 Release date 2024-07-19
*YP 3.1.8 release date 2021/06/04
*YP 5.1 M3 Build date 2024-08-26
*YP 3.1.9 build date 2021/06/21
*YP 5.1 M3 Release date 2024-09-06
*YP 3.1.9 release date 2021/07/02
*YP 5.1 M4 Build date 2024-09-30
*YP 3.3.2 build date 2021/07/19
*YP 5.1 M4 Release date 2024-10-25
*YP 3.3.2 release date 2021/07/30
*YP 3.1.10 build date 2021/07/26
*YP 3.1.10 release date 2021/08/06
*YP 3.1.11 build date 2021/09/13
*YP 3.1.11 release date 2021/9/24


'''Tracking Metrics:'''<br/>
'''Upcoming dot releases:'''<br/>
*WDD 2664 (last week 2674) (https://wiki.yoctoproject.org/charts/combo.html)
*YP 4.0.19 is in QA.
*Poky Patch Metrics 
*YP 5.0.2 Build Date 2024-06-24
**Total patches found: 1297 (last week 1331)
*YP 5.0.2 Release Date 2024-07-05
**Patches in the Pending State: 494 (38%) [last week 497 (37%)]
*YP 4.0.20 Build Date 2024-07-15
*YP 4.0.20 Release Date 2024-07-26
*YP 5.0.3 Build Date 2024-08-12
*YP 5.0.3 Release Date 2024-08-23
*YP 4.0.21 Build Date 2024-09-09
*YP 4.0.21 Release Date 2024-09-20
*YP 5.0.4 Build Date 2024-09-23
*YP 5.0.4 Release Date 2024-10-04
*YP 4.0.22 Build Date 2024-10-14
*YP 4.0.22 Release Date 2024-10-25
*YP 5.0.5 Build Date 2024-11-11
*YP 5.0.5 Release Date 2024-11-22
*YP 4.0.23 Build Date 2024-11-18
*YP 4.0.23 Release Date 2024-11-29


'''The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:'''<br/>
'''The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:'''<br/>
Line 65: Line 76:


== Archives ==
== Archives ==
*[[2024 Yocto Project Weekly Status Archive]]
*[[2023 Yocto Project Weekly Status Archive]]
*[[2022 Yocto Project Weekly Status Archive]]
*[[2021 Yocto Project Weekly Status Archive]]
*[[2021 Yocto Project Weekly Status Archive]]
*[[2020 Yocto Project Weekly Status Archive]]
*[[2020 Yocto Project Weekly Status Archive]]
*[[2019 Yocto Project Weekly Status Archive]]
*[[2019 Yocto Project Weekly Status Archive]]
*[[2018 Yocto Project Weekly Status Archive]]
*[[2018 Yocto Project Weekly Status Archive]]

Latest revision as of 14:49, 4 June 2024

Yocto Project Weekly Status June 4th, 2024

Current Dev Position: YP 5.1 M1
Next Deadline: YP 5.1 M1 Build 20 May 2024

Next Team Meetings:

Key Status/Updates:

  • YP 4.0.19 is in QA
  • YP 5.1 M1 rc1 is building
  • We identified the performance problem in bitbake using hash equivalence servers and have made changes to batch queries and improve build times
  • Extra logging for that performance problem is available by setting BB_LOGCONFIG=XXX/bitbake/contrib/autobuilderlog.json (the more verbose logging the autobuilder uses) which will then show if initial query or rehash looping takes longer than 30s for iterations or 60s overall.
  • We changed locked-sigs to include the hash equivalence mapping data, removing the need to copy the cache file.
  • Libtool was upgraded to the 2.5.0 alpha release, many patches were merged upstream.
  • Qemu has been upgraded to 9.0.0, kernel headers to 6.9 and stable and dev kernel updates merged (to 6.6.32)
  • There is one remaining issue on fedora40, systemtap-native does not build. The other Fedora 40 are believed to be fixed.
  • Some layers are setting fallbacks for UNPACKDIR which break OE-Core by trying to claim compatibility with everything. Errors have been added to make this problem clear.
  • sstate has had changes to fix a bug where build-history data was inconsistent. The siteconfig class was removed to make that change easier and improve the code, it was mostly unused anyway.
  • The parallelism in the hash client was removed as batching queries is much more effective and easier on the server.
  • There is an open letter the project has created related to the CVE/NVD situation, more information is available here: https://lists.openembedded.org/g/openembedded-architecture/message/1990
  • Please consider signing this to show support for those changes.
  • Signatures from small businesses/consultancies are as welcome as those from larger organizations as we want to demonstrate the breadth of the need for these changes.
  • There is a further update from Marta/Syslinbit on the CVE tooling/patches
  • We continue to have trouble tracking down the rust reproducibility problem in rustdoc.
  • Post release upgrading is resulting in an improvement in patch metrics which is great to see, more should appear next week in the statistics too.

Ways to contribute:

Tracking Metrics:

YP 5.1 Milestone Dates:

  • YP 5.1 M1 Build date 2024-05-20
  • YP 5.1 M1 Release date 2024-05-31
  • YP 5.1 M2 Build date 2024-07-08
  • YP 5.1 M2 Release date 2024-07-19
  • YP 5.1 M3 Build date 2024-08-26
  • YP 5.1 M3 Release date 2024-09-06
  • YP 5.1 M4 Build date 2024-09-30
  • YP 5.1 M4 Release date 2024-10-25

Upcoming dot releases:

  • YP 4.0.19 is in QA.
  • YP 5.0.2 Build Date 2024-06-24
  • YP 5.0.2 Release Date 2024-07-05
  • YP 4.0.20 Build Date 2024-07-15
  • YP 4.0.20 Release Date 2024-07-26
  • YP 5.0.3 Build Date 2024-08-12
  • YP 5.0.3 Release Date 2024-08-23
  • YP 4.0.21 Build Date 2024-09-09
  • YP 4.0.21 Release Date 2024-09-20
  • YP 5.0.4 Build Date 2024-09-23
  • YP 5.0.4 Release Date 2024-10-04
  • YP 4.0.22 Build Date 2024-10-14
  • YP 4.0.22 Release Date 2024-10-25
  • YP 5.0.5 Build Date 2024-11-11
  • YP 5.0.5 Release Date 2024-11-22
  • YP 4.0.23 Build Date 2024-11-18
  • YP 4.0.23 Release Date 2024-11-29

The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:
https://wiki.yoctoproject.org/wiki/TSC

Archives

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Weekly_Status&oldid=86345"