User:Tracy Graydon/Release Process: Difference between revisions

From Yocto Project
Jump to navigationJump to search
 
(26 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Yocto Release Procedures ==
= Yocto Release Procedures =


This document explains the steps in publishing an official Yocto Project release. This is intended to be a HOWTO Release Guide.
This document explains the steps in publishing an official Yocto Project release. This is intended to be a HOWTO Release Guide.
Line 24: Line 24:
* Sync to external mirrors.
* Sync to external mirrors.
* Tag the git repos.
* Tag the git repos.
* Publish the documentation pages.
* Publish the yoctoproject.org pages.
* Test the links to make sure everything works.
* Test the links to make sure everything works.
* Announce the release.
* Announce the release.


== Release Matrix ==


For Major, Minor, and Point releases, the release artifacts and collateral are the same. For Milestone releases, we publishing is minimal. The table below shows what is published for each release type.


===== Release of Build =====
{| class="wikitable" border="2"
* Run the release script to stage the release. http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/tree/bin/release_scripts/release.py. This script will do the following...
!
    - Create the staging directory.
!Major/Minor
    - Remove the rpm/deb directories.
!Point
    - Delete duplicate files and extraneous blobs, and convert symlinks.
!Milestone
    - Rename tarballs to reflect current naming conventions. For example. poky-<githash>.tar.bz2 should be renamed to poky-<release_name>-<release_number>.tar.bz2
|-
    - Generate the BSP tarballs
|Yoctoproject.org Docs
    - Extract the eclipse-plugin archive to http://downloads.yoctoproject.org/releases/eclipse-plugin/<M>.<m> where M.m is the Yocto version
|Yes
    - Copy the adt-dev repo to the live site (For release lines prior to 2.0.)
|Yes
    - Create the md5sum table for the release
|No
* Copy the staging directory to the downloads directory  
|-
* gpg sign the md5sum table
|YoctoProject.org Release Page
* Create release notes, finalize, get approval, and gpg sign.
|Yes
* Create the release and BSP pages on yoctoproject.org
|Yes
* Create the documentation links for the release on yoctoproject.org
|No
* Sync the release out to the mirrors.
|-
* Once sync out is complete, verify that pages work, and download and documentation links work.
|BSP Pages
* Tag the repos.
|Yes
* Verify the items on the RELEASE CHECKLIST: https://wiki.yoctoproject.org/wiki/Yocto_Project_Release_Checklist
|Yes
* Post release announcement on Blog
|No
* Post release announcement on mailing lists (yocto and yocto-announce)
|-
* Update DISTRO and DISTRO_VERSION in master to be the main release+snapshot
|Eclipse Plugin Pages
|Yes
|Yes
|No
|-
|BuildAppliance Page
|Yes
|Yes
|No
|-
|Buildtools Page
|Yes
|Yes
|No
|-
|Release Notes (signed)
|Yes
|Yes
|No
|-
|Tag Release
|Yes
|Yes
|Yes
|-
|Email Announcement
|Yes
|Yes
|Yes
|}
 
== Staging the Release ==
 
Most of the release process, aside from the creation of the YoctoProject.org pages, is done via scripts. These scripts live here:
http://git.yoctoproject.org/cgit/cgit.cgi/release-tools/
 
The release.py script does the bulk of the staging work based on release type. You will need to know whether you are releasing a Major, Minor, Point, or Milestone release. You will also need to know the release codename (i.e. Krogoth, Morty, Pyro, Foo) and the poky release number. i.e. Pyro is 17.0.0.
 
The release codename, YP version (i.e. 2.3), and Poky version (16.0, 17.0, etc) can be found here: https://wiki.yoctoproject.org/wiki/Releases
 
To be able to perform a release, you will need the appropriate access to various hosts, and the requisite permissions to execute a release. You will need ssh access to a suitable host system that has access to the NAS. For example, one of the autobuilder buildslaves would be appropriate. Michael Halstead is the contact person for such access. Michael Halstead <mhalstead@linuxfoundation.org> #halstead on freenode.
 
To begin the staging, you will need to be logged in as the pokybuild user. You will need the aforementioned scripts. Namely, you will need release.py and, if doing a minor or point release, you will also need the relnotes.py script. We do not generate release notes for Milestones. Paul Eggleton generates the most glorious release notes you have ever seen for Major releases. Not kidding. They are a work of art.
 
Release candidates land in https://autobuilder.yoctoproject.org/pub/releases/
 
I will use yocto-2.3.rc4 as an example in the steps that follow. Note that all release candidates will end with ".rc<#>". This is a Minor release, whose codename is Pyro and Poky version is 17.0.0.
 
==== Stage the All the Things ====
 
* ssh into your chosen system
* sudo -iu pokybuild
* Snag a copy of the release tools: https://git.yoctoproject.org/git/release-tools
* Identify your release candidate. As noted above, you need the following info:
** Release version (2.3)
** Poky version (17.0.0)
** Codename (pyro)
* Run the release.py script. In this example: python release.py -i yocto-2.3.rc4 -p 17.0.0 -b pyro
** For a milestone release: python release.py -i yocto-2.3_M3.rc2  You do not need poky version nor the codename for a milestone.
** python release.py --help  will display your options.
* Go to dinner. Go to bed. Come back in the morning.
 
Not kidding. This takes many, many hours. The release script first makes a copy of the RC directory, where all work takes place. We do NOT touch the RC directory AT ALL. It will make you cry if you have to try to restore something that gets hosed in there. Seriously. Don't touch it.
 
The staging directory will have the same name, sans the ".rc<#>" portion. In this example, the staging directory will be created in the same top level dir as the candidate, and be called "yocto-2.3."  i.e. https://autobuilder.yoctoproject.org/pub/releases/yocto-2.3/  Note this link may or may not exist at the time of reading. We clean up old staging and RC directories once releases are published and we know all is well. It is intended for an example only.
 
After copying over the content of the RC directory to the staging directory, a good deal of clean up occurs. This involves such things as deleting files we do not publish with releases, removing duplicate files, and converting symlinks to actual files. All of which is very time consuming.
 
== Release Notes ==
 
===== General Process =====
 
If staging a Major or Minor release (2.0, 2.3, etc.) release notes will come to you from on high, via Paul Eggleton.  
 
If this is a point release (i.e. 2.2.1 or 2.3.1, etc) you will need to run the relnotes.py script to generate the release notes template and send it as a draft to the CCB list and any other relevant contributors. Known Issues get added, corrections made, etc., or not, and the release notes are finalized.  
 
 
* Run the relnotes.py script
:<actual example>
:<comments about revisions to use>
:<what to do with it>
 
== YoctoProject.org Pages ==
 
:<stuff>
:<do this, do that>
:<It wasn't my idea, don't blame me>
:<how to add the taxonomy/structure for the new release>
:<list of docs to clone>
:<list of pages to clone>
:<invoke the Release Matrix>
:<unpublished to published when going live>
:<release notes come back to haunt us>
 
 
== Stage for Sync to External Mirrors ==
 
Which is a hideous subtitle that needs to be fixed.
 
:<rsync to downloads dir>
:<sign the release notes and copy to downloads dir (Michael usually moves these as part of the signing process)>
:<verify that external sync has started, else kick it off>
:<whine at Michael if something goes wrong with it.>
 
== Stage Release Announcement Email ==
 
:<get it ready to go>
:<Give the hashes to Michael for the repo tagging>
:<Plain text; watch out for line wrapping>
 
== Tagging =
 
== Sanity Check ==
 
:<what did we miss or hork?>
 
== Announce ==
 
:<once ccb approves, let her rip>

Latest revision as of 07:44, 3 May 2017

Yocto Release Procedures

This document explains the steps in publishing an official Yocto Project release. This is intended to be a HOWTO Release Guide.

Yocto Project Naming Conventions

Yocto Project releases fall into four main types:

  • Major Ex. yocto-2.0
  • Minor Ex. yocto-2.1
  • Point (aka "patch" release) Ex. yocto-2.1.1
  • Milestone Ex. yocto-2.3_M3

A full explanation of release naming conventions, release types, and background can be found here: https://wiki.yoctoproject.org/wiki/Yocto_Project_Release_Process

General Steps for a Release:

  • Stage the release candidate.
  • Generate or obtain release notes, if applicable.
  • Stage/create the Release documentation on YoctoProject.org (unpublished but ready to go).
  • Sync the release to the downloads area in preparation for sync to external mirrors.
  • Sign the release notes.
  • Sync to external mirrors.
  • Tag the git repos.
  • Publish the yoctoproject.org pages.
  • Test the links to make sure everything works.
  • Announce the release.

Release Matrix

For Major, Minor, and Point releases, the release artifacts and collateral are the same. For Milestone releases, we publishing is minimal. The table below shows what is published for each release type.

Major/Minor Point Milestone
Yoctoproject.org Docs Yes Yes No
YoctoProject.org Release Page Yes Yes No
BSP Pages Yes Yes No
Eclipse Plugin Pages Yes Yes No
BuildAppliance Page Yes Yes No
Buildtools Page Yes Yes No
Release Notes (signed) Yes Yes No
Tag Release Yes Yes Yes
Email Announcement Yes Yes Yes

Staging the Release

Most of the release process, aside from the creation of the YoctoProject.org pages, is done via scripts. These scripts live here: http://git.yoctoproject.org/cgit/cgit.cgi/release-tools/

The release.py script does the bulk of the staging work based on release type. You will need to know whether you are releasing a Major, Minor, Point, or Milestone release. You will also need to know the release codename (i.e. Krogoth, Morty, Pyro, Foo) and the poky release number. i.e. Pyro is 17.0.0.

The release codename, YP version (i.e. 2.3), and Poky version (16.0, 17.0, etc) can be found here: https://wiki.yoctoproject.org/wiki/Releases

To be able to perform a release, you will need the appropriate access to various hosts, and the requisite permissions to execute a release. You will need ssh access to a suitable host system that has access to the NAS. For example, one of the autobuilder buildslaves would be appropriate. Michael Halstead is the contact person for such access. Michael Halstead <mhalstead@linuxfoundation.org> #halstead on freenode.

To begin the staging, you will need to be logged in as the pokybuild user. You will need the aforementioned scripts. Namely, you will need release.py and, if doing a minor or point release, you will also need the relnotes.py script. We do not generate release notes for Milestones. Paul Eggleton generates the most glorious release notes you have ever seen for Major releases. Not kidding. They are a work of art.

Release candidates land in https://autobuilder.yoctoproject.org/pub/releases/

I will use yocto-2.3.rc4 as an example in the steps that follow. Note that all release candidates will end with ".rc<#>". This is a Minor release, whose codename is Pyro and Poky version is 17.0.0.

Stage the All the Things

  • ssh into your chosen system
  • sudo -iu pokybuild
  • Snag a copy of the release tools: https://git.yoctoproject.org/git/release-tools
  • Identify your release candidate. As noted above, you need the following info:
    • Release version (2.3)
    • Poky version (17.0.0)
    • Codename (pyro)
  • Run the release.py script. In this example: python release.py -i yocto-2.3.rc4 -p 17.0.0 -b pyro
    • For a milestone release: python release.py -i yocto-2.3_M3.rc2 You do not need poky version nor the codename for a milestone.
    • python release.py --help will display your options.
  • Go to dinner. Go to bed. Come back in the morning.

Not kidding. This takes many, many hours. The release script first makes a copy of the RC directory, where all work takes place. We do NOT touch the RC directory AT ALL. It will make you cry if you have to try to restore something that gets hosed in there. Seriously. Don't touch it.

The staging directory will have the same name, sans the ".rc<#>" portion. In this example, the staging directory will be created in the same top level dir as the candidate, and be called "yocto-2.3." i.e. https://autobuilder.yoctoproject.org/pub/releases/yocto-2.3/ Note this link may or may not exist at the time of reading. We clean up old staging and RC directories once releases are published and we know all is well. It is intended for an example only.

After copying over the content of the RC directory to the staging directory, a good deal of clean up occurs. This involves such things as deleting files we do not publish with releases, removing duplicate files, and converting symlinks to actual files. All of which is very time consuming.

Release Notes

General Process

If staging a Major or Minor release (2.0, 2.3, etc.) release notes will come to you from on high, via Paul Eggleton.

If this is a point release (i.e. 2.2.1 or 2.3.1, etc) you will need to run the relnotes.py script to generate the release notes template and send it as a draft to the CCB list and any other relevant contributors. Known Issues get added, corrections made, etc., or not, and the release notes are finalized.


  • Run the relnotes.py script
<actual example>
<comments about revisions to use>
<what to do with it>

YoctoProject.org Pages

<stuff>
<do this, do that>
<It wasn't my idea, don't blame me>
<how to add the taxonomy/structure for the new release>
<list of docs to clone>
<list of pages to clone>
<invoke the Release Matrix>
<unpublished to published when going live>
<release notes come back to haunt us>


Stage for Sync to External Mirrors

Which is a hideous subtitle that needs to be fixed.

<rsync to downloads dir>
<sign the release notes and copy to downloads dir (Michael usually moves these as part of the signing process)>
<verify that external sync has started, else kick it off>
<whine at Michael if something goes wrong with it.>

Stage Release Announcement Email

<get it ready to go>
<Give the hashes to Michael for the repo tagging>
<Plain text; watch out for line wrapping>

= Tagging

Sanity Check

<what did we miss or hork?>

Announce

<once ccb approves, let her rip>