Processes and Activities
Yocto Release Process
Steps to release Yocto:
Run an autobuilder "generate-release" build. This will create images in /srv/www/vhosts/autobuilder.pokylinux.org/generate-release/<datestamp>/. Rename these files with the appropriate release name and version (Scott will have a script to do this by the next release).
For post-Green releases, there will be a release staging area that will be used to mirror the output before the files are released on the public web site.
On release branch:
- Set DISTRO_VERSION in poky.conf
- Update handbook references to stable release (introduction.xml, master branch needs this too)
- Update version reference and updated date in handbook (poky-handbook.xml)
- git commit -a -m "Purple 3.2 Release"
- git tag -a purple-3.2 -m "Tag Purple 3.2"
- git archive purple-3.2 --prefix=purple-3.2/ | bzip2 > /tmp/poky-purple-3.2.tar.bz2 scp /tmp/poky-purple-3.2.tar.bz2 o- hand.com:/srv/www/pokylinux.org/releases/poky-purple-3.2.tar.bz2
On master branch:
- Set DISTRO_VERSION in poky.conf to new version
- Update version reference and updated date in handbook (poky-handbook.xml)
- cd handbook
- make
- scp -r poky-handbook.tgz poky-handbook.html poky-handbook.pdf *.png *.xml *.css *.svg o-hand.com:/srv/www/pokylinux.org/releases/purple-3.2/doc/
- scp -r poky-handbook.tgz poky-handbook.html *.png *.xml *.css *.svg o-hand.com:/srv/www/pokylinux.org/releases/doc/
(or copy across by hand when remotely connected to machine?)
- Edit web pages (website is controlled from a git repo)
- support/index.php
- getit/index.php
- index.php
- poky-wp-theme/common-funcs.inc
- poky-wp-theme/front-page.php
- Post release announcement on Blog
- Post release announcement on mailing list
BKMs for Package Updating
This page will be for capturing the BKMs of package upgrading as we get reviewed and process more of the packages.
Don't Retain older Versions
Unless there is a specific need (which will probably occur for GPLv2, this has also happened in the past when platforms have known bugs in the latest version) don't retain older versions of the recipe files and patches.
Use git mv to rename recipe and patches
From Josh: Generally the way I perform recipe upgrades is to use git mv to rename the old file to the new version, this means that you don't have to consciously delete the old version later (win 1) and that git tracks the rename and the differences with the old file, which doesn't happen with a delete and add (win 2).
The advantage of this is that you and any reviewers can more easily see what has changed with the updated version of the recipe.
Nitin: Actually {git mv a b} is nothing but {mv a b; git rm a; git add b} And the file renames are detected by git automatically by looking at the file contents. Because of this git behavior, git shows some renames as renames and sometimes not. And I did not find a way to force git to understand renames. So if after "git mv" git is showing "file add & file remove", then don't get surprised, it's normal git operation.
Reset PR to 0 (or add PR) when upgrading recipes
This is a good visual reminder to bump in the future if needed.
For upgrades it's ok to use OE for inspiration
When upgrading packages, do the git mv as above, and build if it breaks, it's OK to check the OE equivalent and grab new patch or configuration change, just don't grab the whole recipe.
Be sure to add a credit to OE in your commit message when you do take a change like this.
Review: Merging packages from OpenEmbedded
For new package grab OE version, but sanitize them
Follow the recipe rules for Yocto when you take an OE recipe, be sure to clean all the OE'isms out
Again, give credit to OE in commit messages
Review: Merging packages from OpenEmbedded
Do NOT Merge .inc / .bb files
This was a decision that slipped by me, and was not communicated well, we do not want to merge the common ".inc" files and recipe (.bb) files. This is the case even if the .bb only contains a require and a PR="r0" line. This does not mean split everything at this point, but don't merge going forward.
White Space Management
- Most variables such as SRC_URI should use spaces.
- Shell functions should use tabs
- Python functions should use spaces (4 spaces per indent).
Commenting in Pacthes
When you change or add patches, be sure to include attributions of where the patch came from, along with your full name and email similar to "Signed-off-by", no special tag is needed (yet?), but having your full name and email is important to track it.
New style patch application
The patch and pnum parameters have been renamed to the more logical apply and striplevel. The apply parameter takes either "yes" or "no" and the striplevel parameter takes an integer (0, 1, etc).
Both parameters are now optional with "sane" defaults.
The apply parameter is optional for SRC_URI lines with patch or diff extensions, which will default to being applied.
The striplevel parameter is also optional with a default striplevel of 1.
Old style parameters (patch and pnum) will continue to work for some time but it would be useful to move to the new style syntax as people are updating other parts of their recipes.
Therefore a patch line would be changed from:
file://some.patch;patch=1;pnum=2
to:
file://some.patch;striplevel=2
and a patch line:
file://another.diff;patch=1;pnum=1
could be changed to:
file://another.diff
SDK Generator
SDK Userspace NFS Use Scenarios and Workflow
Scenario 1: Minimal installation using meta-toolchain with a QEMU SDK image
Sally is an embedded application developer who works for a large corporation which locks-down its developer workstations and does not allow developers root access. :(
Sally's System Administrator set up her workstation for Poky development by doing the following:
- Extracting the output from a meta-toolchain build (poky-glibc-x86_64-i586-toolchain-XYZ.tar.bz2) into /opt/poky
- Install and ensure that rpcbind or portmap is running (note: rpcbind requires an insecure option, -i to work)
- Run the poky-qemu-ifup command N times to create a bank of tun devices that can be used by QEMU networking. These devices are owned by a group that Sally is a member of.
Sally then sources the meta-toolchain environment file (source /opt/poky/environment-setup-i586-poky-linux), which adds the toolchain's usr/bin directory into her $PATH and sets some other build-related environment variables.
She then downloads a poky-image-sdk tarball and a QEMU kernel. Sally can copy the kernel and extract the SDK image tarball to an arbitrary work area, but she must extract the SDK image tarball using the following script:
poky-extract-sdk <poky-sdk-tarball> <extract-dir>
The documentation may recommend standard directories (e.g, ~/qemukernels/<kernel> ~/rootfs/<machinetype>), but Sally is free to choose another directory if that suits her needs better.
She then can run QEMU to automatically boot to this nfsroot with the following command:
runqemu-nfs {machine-type} <kernel> <rootfs-dir>
Where machine-type is optional if the kernel is named according to poky conventions (e.g, bzImage-qemux86.bin). This script would automatically start up the userspace NFS server using binaries from /opt/poky and create runtime files (such as exports, rmtab, *.pid files, and so on) in ~/.poky-sdk/
It would then start up QEMU with the next available tap network device and mount its rootfs.
When Sally shuts down the QEMU session (either gracefully or by simply killing it), the userspace NFS server would be shut down and the tap network device released automatically. Sally could also forcefully shut down the NFS server and networking with the following command:
poky-export-rootfs stop <path to rootfs>
Scenario 2: In-tree Poky setup with a QEMU SDK image
Roger is an embedded developer who, like Sally, works for a large corporation which locks-down its developer workstations and does not allow developers root access. :(
Roger's System Administrator sets up his workstation for Poky development by doing the following:
- Install and ensure that rpcbind or portmap is running (note: rpcbind requires an insecure option, -i to work)
- Run the poky-qemu-ifup command N times to create a bank of tun devices that can be used by QEMU networking. These devices are owned by a group that Roger is a member of.
Thus, the only difference is that Roger doesn't need the cross-toolchain installed in /opt/poky. That is because Roger is willing to do full Poky builds, and has the Poky tarball extracted in some area he prefers to work (e.g, ~/poky).
Roger sources the poky-init-build-env script, which sets up his environment for Poky builds. He then bitbakes a poky-image-sdk image, which creates a qemu kernel and SDK image tarball in his ~/poky/build/tmp/deploy/images/ directory.
Roger then extracts the SDK image tarball with:
poky-extract-sdk <poky-sdk-tarball> <extract-dir>
and then starts QEMU to boot to this using:
runqemu-nfs {machine-type} <kernel> <rootfs-dir>
Where machine-type is optional if the kernel is named according to poky conventions (e.g, bzImage-qemux86.bin). This script would automatically start up the userspace NFS server using binaries from the automatically detected Poky native sysroot and create runtime files (such as exports, rmtab, *.pid files, and so on) in ~/.poky-sdk/
It would then start up QEMU with the next available tap network device and mount its rootfs.
When Roger shuts down the QEMU session (either gracefully or by simply killing it), the userspace NFS server would be shut down and the tap network device released automatically. Roger could also forcefully shut down the NFS server and networking with the following command:
poky-export-rootfs stop <path to rootfs>
Scenario 3: Exporting a rootfs to a non-QEMU target
This allows developers to export a rootfs to non-qemu targets (i.e, real hardware).
First, the rootfs must be created using the poky-extract-sdk script, which creates the filesystem tree using pseudo:
poky-extract-sdk <poky-sdk-tarball> <extract-dir>
Then they can start/stop/restart the unfs server with:
poky-export-rootfs {start|stop|restart} <path to rootfs>
The developer would then configure their hardware to boot using a kernel accessed over tftp and with a rootfs pointed at the hosts's IP and exported directory.
