Yocto Project Release Process: Difference between revisions

From Yocto Project
Jump to navigationJump to search
 
(39 intermediate revisions by 6 users not shown)
Line 9: Line 9:
* M = major release number
* M = major release number
* m = minor release number
* m = minor release number
* p = minor rev release number
* p = minor rev release number (aka "point" or "patch" release)


Major release number changes imply compatibility changes with previous releases. Minor release number changes imply significant changes up to, but not including compatibility changes. Minor rev number changes are for minor issues such as simple bugfixes, security updates, etc.  
Major release number changes imply compatibility changes with previous releases. Minor release number changes imply significant changes up to, but not including compatibility changes. Minor rev number changes are for minor issues such as simple bugfixes, security updates, etc.  
Line 32: Line 32:


=== Release Matrix ===
=== Release Matrix ===
Yocto Project/Poky Releases, while they do not share the same numbering, are released at the same time. This includes point releases. So, for example, the 1.4.1 Yocto Project point release, was used to build the dylan-9.0.1 poky point release.


{| style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 0"
{| style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 0"
! align="left"| Yocto Project Release
|+Yocto Project/Poky Releases
! Poky Name
! style="border-style: solid; border-width: 0 1px 1px 0" | Yocto Project Release
! Poky Release
! style="border-style: solid; border-width: 0 1px 1px 0" | Poky Release Name
! style="border-style: solid; border-width: 0 1px 1px 0" | Poky Release
|-
|3.0.x
|Zeus
|22.0.x
|-
|2.7.x
|Warrior
|21.0.x
|-
|2.6.x
|thud
|20.0.x
|-
|2.5.x
|sumo
|19.0.x
|-
|2.4.x
|rocko
|18.0.x
|-
|2.3.x
|pyro
|17.0.x
|-
|2.2.x
|monty
|16.0.x
|-
|2.1.x
|krogoth
|15.0.x
|-
|2.0.x
|jethro
|15.0.x
|-
|1.8.x
|fido
|13.0.x
|-
|1.7.x
|dizzy
|12.0.x
|-
|1.6.x
|daisy
|11.0.x
|-
|-
|1.5.x
|1.5.x
Line 49: Line 101:
|danny
|danny
|8.0.x
|8.0.x
|-
|1.2.x
|denzil
|7.0.x
|-
|1.1.x
|edison
|6.0.x
|-
|1.0.x
|bernard
|5.0.x
|-
|0.9.x
|laverne
|4.0.x
|}
|}


== Types of Releases ==
A full listing of releases, code names, Poky and BitBake versions and other related info may be found here:


==== Internal/Nightly Releases ====
https://wiki.yoctoproject.org/wiki/Releases


Nightly releases can be considered a "base class" of the release system. Weekly releases are generated directly from them, and otherwise the nightly release build status serves as a fundamental health status of the builds. Nightly releases are built directly from poky master and '''failures should be immediately addressed or the offending changes reverted'''.
== Types of Releases ==


==== Internal/QA Weekly Releases ====
The day to day testing of the project is now handled through the autobuilder with pre-merge testing of patches.
 
These releases are the basic unit of internal progress for the Yocto project.
 
Every Wednesday (mid-day US Pacific time), the nightly build will generate a release which gets submitted to QA that evening (the start of China's Thursday).
 
If this weekly release passes initial testing from QA, the QA team will continue on to run their full weekly test suite.
 
If the weekly release does ''not'' pass initial testing from QA, the QA team will report this to the distro team and the distro team will focus on addressing these issues the next day. If the distro team lead believes these changes will resolve the reported issues, he/she will submit the nightly build for that day to QA again.
 
Weekly releases will be made available to the QA team via a web-accessible directory on the autobuilder. Within this directory, a tarball of all of the above components will be available to make downloading the release more convenient.  


==== Milestone Releases ====
==== Milestone Releases ====
Line 73: Line 131:
These releases are performed at the end of a milestone period and are used to measure our progress in delivering new features to Yocto Linux.
These releases are performed at the end of a milestone period and are used to measure our progress in delivering new features to Yocto Linux.


To generate a milestone release, a snapshot of poky/meta-intel/eclipse-poky/meta-qt3 master is taken and pushed to a milestone branch. This branch then serves as a stabilization branch, allowing poky master to continue feature development. builds/milestone can then be improved by pulling in fixes via cherry-picking from poky master.
To generate a milestone release, we generate a release candidate from master. After each release candidate has cleared QA, the branch is tagged (M.m_M1).


After each release candidate is generated, the branch is tagged (M.m_M1.rcX). Once stabilization has occurred, the branch is then tagged as M.m_M1.final and git tar archives are generated. No images are released for milestone builds.
The milestone is copied to the release area preserving links and then cleaned up. We remove rpm/deb directories and then remove .gz files in machines (except for p1022), bz2 files (except for p1022) and then all ext3 files for everything except qemu machines.


==== Point releases/Major Releases ====
==== Point releases/Major Releases ====


These are the final, QA-tested, manager-approved releases of Yocto Linux which will '''WOW''' our customers and change the future of embedded Linux devices. Words can barely describe the awesomeness of these SDK and filesystem images!
These are the final, QA-tested, TSC approved releases of the project.


== Release Components ==
== Release Components ==
Line 87: Line 145:
==== Distro Components ====
==== Distro Components ====


* Bootable QEMU images of minimal and sato for the following architectures: qemux86, qemux86-64, qemuarm, qemumips, qemuppc
* Bootable QEMU images of minimal and sato for the following architectures: qemux86, qemux86-64, qemuarm, qemuarm64, qemumips, qemuppc
** A bootable QEMU image consists of a kernel file, a kernel modules tarball, and root filesystem tarball
** A bootable QEMU image consists of a kernel file, a kernel modules tarball, and root filesystem tarball
* Non-emulated machine targets (e.g, netbook, emenlow) will include appropriate images (e.g, live images) of minimal and sato.
* Non-emulated machine targets (e.g, beaglebone-yocto, genericx86, generic86-64, generic-arm64) will include appropriate images (e.g, live images) of minimal and sato.


==== SDK Components ====
==== SDK Components ====


* meta-toolchain tarballs for the following host/target combinations: i586/[arm|i586|mips|ppc|x86_64], x86_64/[arm|i586|mips|ppc|x86_64]
* meta-toolchain tarballs for the following host/target combinations: i586/[arm|i586|mips|ppc|x86_64], x86_64/[arm|i586|mips|ppc|x86_64], arm64
* Bootable SDK images for the following architectures: qemux86, qemux86-64, qemuarm, qemumips, qemuppc
* Bootable SDK images for the following architectures: qemux86, qemux86-64, qemuarm, qemuarm64, qemumips, qemuppc
* SDK plugins for the following IDEs: Anjuta, Eclipse


Currently the SDK plugins are not being generated with the remaining build output. Jessica and Scott will need to define a process for this that makes sense before public release. There are issues to be take into consideration such as how the plugins are distributed (Anjuta and Eclipse have their own plugin repositories, for example).
==== Other components ====


==== Other components ====
* named git archives - a file with each git archived named with the hash of the git commit used the images were built from
* adt repository - A repository of all ipk packages used to build the images (Deprecated as of the 2.0 release. Newer releases now have eSDK instead.)
* RELEASENOTES - a gpg signed file with all bugfixes, added features and git hash info for a release
* *.md5sums - md5sum files of all files within the release
 
== Release Preparation Notes ==


* gitinfo - a file which includes information about which commit the images were built from
Need to ensure we:
* Changelog - a file which includes git log information between the last released version and the newly built version
* Update LAYERSERIES_CORENAMES in meta/conf/layer.conf of OE-Core
* Bugfixes - a list of bugs, extracted from the Changelog, that were fixed between the last released version and the newly built version
* Update LAYERSERIES_COMPAT in meta/conf/layer.conf, meta-skeleton/conf/layer.conf, meta-selftest/conf/layer.conf of OE-Core
* Update LAYERSERIES_COMPAT in conf/layer.conf of meta-gplv2 (not required for yocto-4.1 onward)
* Update LAYERSERIES_COMPAT in conf/layer.conf of meta-mingw
* Add release to scheulers.py in yocto-autobuilder
* Add branch to yocto-autobuilder-helper (usually based on master)
* Copy master branch of yocto-testresults to new release branch
* Copy perf-centos7.yoctoproject.org/master/qemux86 of yocto-buildstats to perf-centos7.yoctoproject.org/<branchname>/qemux86
* Copy perf-ubuntu1604/master/qemux86 of yocto-buildstats to perf-ubuntu1604/<branchname>/qemux86
* Update Poky version (not required for yocto-3.3 onward)
* Update bitbake version
* Update build-appliance SRCREV


== Major/Point Release Procedures ==
== Major/Point Release Procedures ==
Line 119: Line 191:
* Post the pre-written release announcement on the blog and mailing lists
* Post the pre-written release announcement on the blog and mailing lists


==== Steps to building a release for Yocto ====
==== Release Process for the Yocto Project ====
 
===== Prior to last milestone =====
* Release Engineer gets release name from Richard Purdie and announces it via yocto@yoctoproject.org
* Release Engineer informs Documentation of the new release names so that they have time to modify documentation
 
===== Last milestone =====
During the last milestone we generally will build from master and then branch when things are at the point where they're mostly stable. We do this to reduce the amount of work required to maintain two branches. Once we're seeing good daily builds and feel confident enough to branch, we will branch the poky, yocto-docs, meta-yocto, yocto-autobuilder-helper repos to a branch named after the name of the release (dora, dylan, laverne, zeus, etc).


Prior to release build:
===== Prior to release build ===== 
* If the build is a point release, use the appropriate release branch (edison, bernard, etc). If not, create a milestone release branch in the following repositories. The milestone branch follows <Major>.<minor>_M<milestone_number> where Major and minor are Yocto release Major and minor release numbers:
* If the build is a point release, use the appropriate release branch (edison, bernard, etc). If not, create a milestone release branch in the following repositories. The milestone branch follows <Major>.<minor>_M<milestone_number> where Major and minor are Yocto release Major and minor release numbers.
-  poky
-  meta-qt3
-  meta-intel
-  eclipse-plugin
-  meta-x32
* Set DISTRO and DISTRO_VERSION in meta/conf/poky.conf
* Set DISTRO and DISTRO_VERSION in meta/conf/poky.conf
* Update handbook references to stable release (introduction.xml, master branch needs this too)
* Update handbook references to stable release (introduction.xml, master branch needs this too)
Line 135: Line 209:
* Update MIRRORS to use the autobuilders source dir
* Update MIRRORS to use the autobuilders source dir


===== Checklist =====
===== Release of Build =====
* Are docs within the release branch current and correct?
* Run the release script to stage the release. This is not required from Kirkstone onwards. https://git.yoctoproject.org/release-tools/tree/release.py?h=dunfell-ReleaseProcess . This script will do the following...
* Is the tarball location within the release branch correct?
    - Create the staging directory.
* Has DISTRO and DISTRO_VERSION and SDK_VERSION been flipped?
    - Remove the rpm/deb directories.
* Have we branched/tagged poky/meta-intel/meta-qt3 etc?
    - Delete duplicate files and extraneous blobs, and convert symlinks.  
* Do the mirrors work?
    - Rename tarballs to reflect current naming conventions. For example. poky-<githash>.tar.bz2 should be renamed to poky-<release_name>-<release_number>.tar.bz2
 
    - Generate the BSP tarballs
==== Steps to release Yocto ====
    - Extract the eclipse-plugin archive to http://downloads.yoctoproject.org/releases/eclipse-plugin/<M>.<m> where M.m is the Yocto version
 
    - Copy the adt-dev repo to the live site (For release lines prior to 2.0.)
Once a release candidate has been validated as a GO:
    - Create the md5sum table for the release
* git branch the milestone branches as <Yocto name> eg "edison", "bernard" etc.
* Copy the staging directory to the downloads directory (not required for Kirkstone onwards)
* git tag/sign/commit the above repo's release branch HEADs as <Yocto_name>-<M>.<m>.<p> where M.m.p is the Poky version.
* Check the Build log for build issue. Build URL included in build notification email(qa-email).
* Copy http://autobuilder.yoctoproject.org/pub/nightly/<datestamp> to http://downloads.yoctoproject.org/releases/yocto/yocto-<M>.<m>. Set to non-world readable
* Check the testresult and regression report for any issue.
* Remove all timestamp images from machines/*
* run https://git.yoctoproject.org/release-tools/tree/testresults.py to combined the manual test result into yocto-testresults. This will create new commit and tag for yocto-testresults. In certain condition this script might break, this can mean something wrong with any artefacts or yocto-testresults* repo. This will also create testreport.txt
* For final release, copy DL_DIR to yocto_M.m/sources
* Create release notes, finalize, get approval, and gpg sign.
* Modify the yocto tarball so that it extracts to poky-<Yocto-name>-<M>.<m> where M.m is the Poky version.
* create release notes in .rst and send to docs@lists.yoctoproject.org. (for 4.0 onwards)
* Create the ADT repo using the ipk directory
* Check the new documentation for the release is available on docs.yoctoproject.org (for 4.0 onward)
* Extract the eclipse-plugin archive to http://downloads.yoctoproject.org/releases/eclipse-plugin/<M>.<m> where M.m is the Yocto version
* Sync the release out to the mirrors.
* create the md5sum table in http://downloads.yoctoproject.org/releases/yocto/yocto-<M>.<m>
* Once sync out is complete, verify that pages work, and download and documentation links work.
* gpg sign the md5sum table
* Tag the repos.
* create release notes
* Verify the items on the RELEASE CHECKLIST: https://wiki.yoctoproject.org/wiki/Yocto_Project_Release_Checklist
* verify new handbook as been published
* Post release announcement on Blog ( Deprecated ? )  
* Set release directory to world readable. Verify release access.
* Post release announcement on mailing lists (yocto and yocto-announce)
* Post release announcement on Blog
* Update DISTRO and DISTRO_VERSION in master to be the main release+snapshot
* Post release announcement on mailing list
 
On master branch:
* Set DISTRO_VERSION in poky.conf to new version
* Update version reference and updated date in handbook (poky-handbook.xml)
* cd handbook
* make
* scp -r poky-handbook.tgz poky-handbook.html poky-handbook.pdf *.png *.xml *.css *.svg <server>:<path>/releases/purple-3.2/doc/
* scp -r poky-handbook.tgz poky-handbook.html *.png *.xml *.css *.svg <server>:<path>/releases/doc/
(or copy across by hand when remotely connected to machine?)
* Edit web pages (website is controlled from a git repo)  
**support/index.php
**getit/index.php
**index.php
**poky-wp-theme/common-funcs.inc
**poky-wp-theme/front-page.php

Latest revision as of 08:45, 11 December 2023

Yocto Linux Release Engineering Procedures

This document describes release engineering procedures for the Yocto Linux project.

Yocto Project Naming Conventions

Official/public releases will use the following scheme: M.m.p

  • M = major release number
  • m = minor release number
  • p = minor rev release number (aka "point" or "patch" release)

Major release number changes imply compatibility changes with previous releases. Minor release number changes imply significant changes up to, but not including compatibility changes. Minor rev number changes are for minor issues such as simple bugfixes, security updates, etc.

Nightly releases will be named using the following scheme: image-name-M.m.p-date-buildnum.extension, where M.m.p is the release number, where date is the datestamp of the build, e.g, 20100715, and buildnum is a build counter, e.g, 1, 2, 3, in case more than one build is generated the same day.

Milestone releases will be named using the following scheme: image-name-M.m.p-date-milestonenum-buildnum.extension, where the field definitions are the same as above with the addition of milestonenum, e.g. M3.

Our first public release was 0.9.0 in October 2010. Our second public release will be the 1.0.0 release in Spring 2011.

Yocto Project Releases

Yocto Project releases are known by their Major.Minor.patch number. For example:

yocto-1.4.2

The main download directory utilizes this naming convention.

Poky Releases

Poky is the reference distro the Yocto Project releases with each Yocto Project release. These releases are named releases (danny, dora, dylan, edison, etc.) as well as numbered utilizing Major.minor.patch numbering.

Release Matrix

Yocto Project/Poky Releases, while they do not share the same numbering, are released at the same time. This includes point releases. So, for example, the 1.4.1 Yocto Project point release, was used to build the dylan-9.0.1 poky point release.


Yocto Project/Poky Releases
Yocto Project Release Poky Release Name Poky Release
3.0.x Zeus 22.0.x
2.7.x Warrior 21.0.x
2.6.x thud 20.0.x
2.5.x sumo 19.0.x
2.4.x rocko 18.0.x
2.3.x pyro 17.0.x
2.2.x monty 16.0.x
2.1.x krogoth 15.0.x
2.0.x jethro 15.0.x
1.8.x fido 13.0.x
1.7.x dizzy 12.0.x
1.6.x daisy 11.0.x
1.5.x dora 10.0.x
1.4.x dylan 9.0.x
1.3.x danny 8.0.x
1.2.x denzil 7.0.x
1.1.x edison 6.0.x
1.0.x bernard 5.0.x
0.9.x laverne 4.0.x

A full listing of releases, code names, Poky and BitBake versions and other related info may be found here:

https://wiki.yoctoproject.org/wiki/Releases

Types of Releases

The day to day testing of the project is now handled through the autobuilder with pre-merge testing of patches.

Milestone Releases

These releases are performed at the end of a milestone period and are used to measure our progress in delivering new features to Yocto Linux.

To generate a milestone release, we generate a release candidate from master. After each release candidate has cleared QA, the branch is tagged (M.m_M1).

The milestone is copied to the release area preserving links and then cleaned up. We remove rpm/deb directories and then remove .gz files in machines (except for p1022), bz2 files (except for p1022) and then all ext3 files for everything except qemu machines.

Point releases/Major Releases

These are the final, QA-tested, TSC approved releases of the project.

Release Components

Yocto Linux includes a number of software components to be included in each release. These include:

Distro Components

  • Bootable QEMU images of minimal and sato for the following architectures: qemux86, qemux86-64, qemuarm, qemuarm64, qemumips, qemuppc
    • A bootable QEMU image consists of a kernel file, a kernel modules tarball, and root filesystem tarball
  • Non-emulated machine targets (e.g, beaglebone-yocto, genericx86, generic86-64, generic-arm64) will include appropriate images (e.g, live images) of minimal and sato.

SDK Components

  • meta-toolchain tarballs for the following host/target combinations: i586/[arm|i586|mips|ppc|x86_64], x86_64/[arm|i586|mips|ppc|x86_64], arm64
  • Bootable SDK images for the following architectures: qemux86, qemux86-64, qemuarm, qemuarm64, qemumips, qemuppc

Other components

  • named git archives - a file with each git archived named with the hash of the git commit used the images were built from
  • adt repository - A repository of all ipk packages used to build the images (Deprecated as of the 2.0 release. Newer releases now have eSDK instead.)
  • RELEASENOTES - a gpg signed file with all bugfixes, added features and git hash info for a release
  • *.md5sums - md5sum files of all files within the release

Release Preparation Notes

Need to ensure we:

  • Update LAYERSERIES_CORENAMES in meta/conf/layer.conf of OE-Core
  • Update LAYERSERIES_COMPAT in meta/conf/layer.conf, meta-skeleton/conf/layer.conf, meta-selftest/conf/layer.conf of OE-Core
  • Update LAYERSERIES_COMPAT in conf/layer.conf of meta-gplv2 (not required for yocto-4.1 onward)
  • Update LAYERSERIES_COMPAT in conf/layer.conf of meta-mingw
  • Add release to scheulers.py in yocto-autobuilder
  • Add branch to yocto-autobuilder-helper (usually based on master)
  • Copy master branch of yocto-testresults to new release branch
  • Copy perf-centos7.yoctoproject.org/master/qemux86 of yocto-buildstats to perf-centos7.yoctoproject.org/<branchname>/qemux86
  • Copy perf-ubuntu1604/master/qemux86 of yocto-buildstats to perf-ubuntu1604/<branchname>/qemux86
  • Update Poky version (not required for yocto-3.3 onward)
  • Update bitbake version
  • Update build-appliance SRCREV

Major/Point Release Procedures

Release Readiness Review

The Release Readiness Review is a sign-off process for official/public Yocto releases. This process reviews feature completeness, status of open defects, testing status, and verifies the status of documentation. From the release readiness review a decision is made on the status of launching an official release.

Technical Release Procedures

24 hours before the release is made public, the release images would be mirrored to external locations and allow time for the blog and mailing list announcements to be written. The final steps in releasing would be ready to be performed in a matter of seconds:

  • Move the release directory from a staging area into the public web server area
  • Push the web site update including new release information
  • Post the pre-written release announcement on the blog and mailing lists

Release Process for the Yocto Project

Prior to last milestone
  • Release Engineer gets release name from Richard Purdie and announces it via yocto@yoctoproject.org
  • Release Engineer informs Documentation of the new release names so that they have time to modify documentation
Last milestone

During the last milestone we generally will build from master and then branch when things are at the point where they're mostly stable. We do this to reduce the amount of work required to maintain two branches. Once we're seeing good daily builds and feel confident enough to branch, we will branch the poky, yocto-docs, meta-yocto, yocto-autobuilder-helper repos to a branch named after the name of the release (dora, dylan, laverne, zeus, etc).

Prior to release build
  • If the build is a point release, use the appropriate release branch (edison, bernard, etc). If not, create a milestone release branch in the following repositories. The milestone branch follows <Major>.<minor>_M<milestone_number> where Major and minor are Yocto release Major and minor release numbers.
  • Set DISTRO and DISTRO_VERSION in meta/conf/poky.conf
  • Update handbook references to stable release (introduction.xml, master branch needs this too)
  • Update version reference and updated date in handbook (poky-handbook.xml)
  • Run the nightly build using the release branch from the autobuilder. This will create images at http://autobuilder.yoctoproject.org/pub/nightly/<datestamp>/.
  • If the build is "good", and adequate for delivery to QA, git tag/sign/commit the above repo's release branch HEADs as <Major>.<minor>_M<milestone_number>.rc<release_candidate_number>
  • Update MIRRORS to use the autobuilders source dir
Release of Build
   - Create the staging directory.
   - Remove the rpm/deb directories. 
   - Delete duplicate files and extraneous blobs, and convert symlinks. 
   - Rename tarballs to reflect current naming conventions. For example. poky-<githash>.tar.bz2 should be renamed to poky-<release_name>-<release_number>.tar.bz2
   - Generate the BSP tarballs
   - Extract the eclipse-plugin archive to http://downloads.yoctoproject.org/releases/eclipse-plugin/<M>.<m> where M.m is the Yocto version
   - Copy the adt-dev repo to the live site (For release lines prior to 2.0.)
   - Create the md5sum table for the release
  • Copy the staging directory to the downloads directory (not required for Kirkstone onwards)
  • Check the Build log for build issue. Build URL included in build notification email(qa-email).
  • Check the testresult and regression report for any issue.
  • run https://git.yoctoproject.org/release-tools/tree/testresults.py to combined the manual test result into yocto-testresults. This will create new commit and tag for yocto-testresults. In certain condition this script might break, this can mean something wrong with any artefacts or yocto-testresults* repo. This will also create testreport.txt
  • Create release notes, finalize, get approval, and gpg sign.
  • create release notes in .rst and send to docs@lists.yoctoproject.org. (for 4.0 onwards)
  • Check the new documentation for the release is available on docs.yoctoproject.org (for 4.0 onward)
  • Sync the release out to the mirrors.
  • Once sync out is complete, verify that pages work, and download and documentation links work.
  • Tag the repos.
  • Verify the items on the RELEASE CHECKLIST: https://wiki.yoctoproject.org/wiki/Yocto_Project_Release_Checklist
  • Post release announcement on Blog ( Deprecated ? )
  • Post release announcement on mailing lists (yocto and yocto-announce)
  • Update DISTRO and DISTRO_VERSION in master to be the main release+snapshot
Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Yocto_Project_Release_Process&oldid=86127"