Processes and Activities: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
No edit summary
Line 3: Line 3:
* [[BKMs for Package Updating]]
* [[BKMs for Package Updating]]
* [[SDK Generator]]
* [[SDK Generator]]
==Scenario 1: Minimal installation using meta-toolchain with a QEMU SDK image==
Sally is an embedded application developer who works for a large corporation which locks-down its developer workstations and does not allow developers root access. :(
Sally's System Administrator set up her workstation for Poky development by doing the following:
# Extracting the output from a meta-toolchain build (poky-glibc-x86_64-i586-toolchain-XYZ.tar.bz2) into /opt/poky
# Install and ensure that rpcbind or portmap is running (note: rpcbind requires an insecure option, -i to work)
# Run the poky-qemu-ifup command N times to create a bank of tun devices that can be used by QEMU networking. These devices are owned by a group that Sally is a member of.
Sally then sources the meta-toolchain environment file (source /opt/poky/environment-setup-i586-poky-linux), which adds the toolchain's usr/bin directory into her $PATH and sets some other build-related environment variables.
She then downloads a poky-image-sdk tarball and a QEMU kernel. Sally can copy the kernel and extract the SDK image tarball to an arbitrary work area, but she must extract the SDK image tarball using the following script:
poky-extract-sdk <poky-sdk-tarball> <extract-dir>
The documentation may recommend standard directories (e.g, ~/qemukernels/<kernel> ~/rootfs/<machinetype>), but Sally is free to choose another directory if that suits her needs better.
She then can run QEMU to automatically boot to this nfsroot with the following command:
runqemu-nfs {machine-type} <kernel> <rootfs-dir>
Where machine-type is optional if the kernel is named according to poky conventions (e.g, bzImage-qemux86.bin). This script would automatically start up the userspace NFS server using binaries from /opt/poky and create runtime files (such as exports, rmtab, *.pid files, and so on) in ~/.poky-sdk/
It would then start up QEMU with the next available tap network device and mount its rootfs.
When Sally shuts down the QEMU session (either gracefully or by simply killing it), the userspace NFS server would be shut down and the tap network device released automatically. Sally could also forcefully shut down the NFS server and networking with the following command:
poky-export-rootfs stop <path to rootfs>


==QA==
==QA==

Revision as of 20:37, 22 October 2010

Welcome to the Processes and Activities Page!

QA

Milestone Test Report for Yocto

Fullpass Test Report for M3-20100911

Fullpass Test Report for M3-20100911

Sanity Test Report for M3-20100911

Sanity Test Report for M3-20100911

Sanity Test Report for M3-20100909

Sanity Test Report for M3-20100911

Partial Sanity Test Report for M3 20100905

Partial Test Report for M3 20100905

Fullpass Test for M2-20100731

Fullpass Test Report for M2-20100731

M2-20100725 & Bug Update for M2-20100728

I finished testing on QEMU/Netbook for M2-20100725 and verified some bugs with M2-20100728 build. Against M2-20100725 build, I reported 12 new bugs and verified 2 bugs as fixed. With the second build M2-20100728, I verified old critical bugs and reported another 2 new bugs. 2 old bugs are verified as fixed with this build.

M3 Test Execution Plan

M3 fullpass test plan.

M4 Test Execution Plan

M4 fullpass test plan.


There are still 5 open bugs for M2, 4 of them are CRITICAL. And 23 open bugs are marked to M3. As a summary, QEMU targets (x86/arm/mips/ppc) can work without critical bugs. Toolchain has one critical bug. Meanwhile, emenlow/netbook have critical issue with/within X environment.


Open Bugs for M2:

Other Open Bugs reported, target for M3 or later:

Verified Fixed Issues:

Existing Old Issues:

M1 Test Report 2010/7/10

Summary & Issue List

I finished sanitytest for M1 20100710 build. There is 2 new issues found in this testing. qemuppc boot up issue is fixed but qemu will report OOM error when booting qemuppc images. There is no mouse action response after X startup with qemumips images. qemuppc and qemuarm cannot shutdown entirely. There are totally 4 open bugs in this testing.

BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).

Issue List:

Test Result Matrix:

SanityTest Result

Machine Image Boot Network(ping) X startup SSH Service Shutdown Comments
qemux86 poky-image-sato PASS PASS PASS PASS PASS
poky-image-sdk PASS PASS PASS PASS PASS
qemuxarm poky-image-sato PASS PASS PASS PASS FAIL
poky-image-sdk PASS PASS PASS PASS FAIL
qemumips poky-image-sato PASS PASS PASS PASS PASS No mouse action response, some icons broken
poky-image-sdk PASS PASS PASS PASS PASS No mouse action response, some icons broken
qemuppc poky-image-sato PASS PASS N/A PASS FAIL
poky-image-sdk PASS PASS N/A PASS FAIL

Detail Test Result

Test Result can be found here Milestone Test Result for 20100710

Testing Env

Host: Ubuntu 10.04 LTS
Kernel: 2.6.32-21-generic-pae
Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)
Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100710-1/


M1 Test Report 2010/7/8

Summary & Issue List

I finished sanitytest for M1 20100708 build. There are 3 issues found in this testing. The major issue is that there is no cursor in qemuarm and qemumips images. I also tried qemuppc because there is some fixing in latest tree, but qemuppc still fails to startup. 5 issues found in last time testing are fixed. Qemuarm can boot up now. Qemuarm and qemumips can start X now.

BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).

Issue List:

Test Result Matrix:

SanityTest Result

Machine Image Boot Network(ping) X startup SSH Service Shutdown Comments
qemux86 poky-image-sato PASS PASS PASS PASS PASS
poky-image-sdk PASS PASS PASS PASS PASS
qemuxarm poky-image-sato PASS PASS PASS PASS FAIL No cursor after X startup
poky-image-sdk PASS PASS PASS PASS FAIL No cursor after X startup
qemumips poky-image-sato PASS PASS PASS PASS PASS No cursor after X startup
poky-image-sdk PASS PASS PASS PASS PASS No cursor after X startup

Detail Test Result

Test Result can be found here Milestone Test Result for 20100708

Testing Env

Host: Ubuntu 10.04 LTS
Kernel: 2.6.32-21-generic-pae
Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)
Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100708-1/

M1 Test Report 2010/6/29

Summary & Issue List

There are 8 target images be tested, only 3 ones can boot up, while SDK image for qemumips is not built out. Testing for qemuarm and qemuppc is blocked. Network in qemux86 image is not enabled. X failure is still within SDK image. And there is no cursor within qemux86 poky-image-sato. Detail issue list as below.

For BAT testing, LTP and POSIX testing are performed against qemux86 poky-image-sdk and qemumips poky-image-sato. Compared with last time test result, LTP result for qemux86 is same, while POSIX result has one more failure.

BTW, I build qemu-native by myself from poky WR’s branch (git.pokylinux.org:poky-wr). These qemu startup scripts, like “runqemu”, also come from the branch.

Issue List:

  1. No network in qemux86 image, (sato/sdk). It should be caused by lack of e1000 module built in WR kernel.
  2. No cursor after X startup in qemux86 poky-image-sato image.
  3. Qemux86 poky-image-sdk image fails to start X. (Known issue as Saul pointed)
  4. Fail to boot up qemuarm images, (sato/sdk). It seems no root partition when booting. See attachment “qemuarm-poky-image.log” for full log when booting QEMU. qemuarm log
  5. Fail to boot up qemuppc images, (sato/sdk). qemu reports “could not load PPC PREP bios ‘powerpc_rom.bin’”. See attachment “qemuppc-poky-image.log” for full log. qemuppc log
  6. By default, script “runqemu” does not start X for qemuarm, qemuppc and qemumips. I find there is no wnidow popup even I remove the X disable option, "-nographic" for qemumips.

Test Result Matrix:

SanityTest Result

Machine Image Boot Network(ping) X startup SSH Service Shutdown Comments
qemux86 poky-image-sato PASS FAIL PASS N/T PASS No cursor after X startup with show-cursor option
poky-image-sdk PASS FAIL FAIL N/T PASS
qemuxarm poky-image-sato FAIL BLOCK BLOCK BLOCK BLOCK
poky-image-sdk FAIL BLOCK BLOCK BLOCK BLOCK
qemumips poky-image-sato PASS PASS N/A PASS PASS option "-nographic" is set in script "poky-qemu-internal" for qemumips. There is no window popup even I remove the option.
poky-image-sdk N/A N/A N/A N/A N/A No SDK images for qemumips
qemuppc poky-image-sato FAIL BLOCK BLOCK BLOCK BLOCK
poky-image-sdk FAIL BLOCK BLOCK BLOCK BLOCK

BAT Result:

Machine Image LTP(FAIL/TOTAL) POSIX(FAIL/UNSUPPORT/TOTAL)
qemux86 poky-image-sdk 46/1120 51/99/1644
qemumips poky-image-sato 58/1116 N/A

Detail Test Result

Test Result can be found here Milestone Test Result for 20100629

Testing Env

Host: Ubuntu 10.04 LTS
Kernel: 2.6.32-21-generic-pae
Poky gitinfo(used to build qemu-native): 5280cfe8c9cd9c9a880683186c77c24d380978ed (git.pokylinux.org:poky-wr)
Build Image Download URL: Except for SDK images for qemuppc and qemuarm (from http://autobuilder.pokylinux.org/nightly/20100629-1/), all the other images come from http://autobuilder.pokylinux.org/nightly/20100628-1/.

Enable Automation Test in Poky

Currently we have implemented two automated sanity test cases in Poky. User can follow below steps to enable the automation test:

  • Make sure your working tree has latest test code. The first sanity test implementation is checked in poky master tree as commit, fd1d661b391dfad1edaac937a17c1165f635031a.
  • Make sure your machine has package "expect" installed. The package is necessary for testing. We will add a recipe for it later. Currently, you need install it by yourself. On Ubuntu host, you can use apt-get to install expect:
$ apt-get install expect
  • Edit /etc/sudoers to allow the user run sudo without prompted for a password. For example, you can do like following for user "tester":
$ cat /etc/sudoers
# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the man page for details on how to write a sudoers file.
#
....
....
....

%tester ALL=(ALL) NOPASSWD: NOPASSWD: ALL
  • Typically, you need a X environment to start QEMU. You can use your desktop's DISPLAY. Or, alternatively, you can install vncserver, and startup a X environment in localhost:x.0. (x can be 1, 2, 3.....n)
  • You need set "IMAGETEST" to "qemu" in build/conf/local.conf
$ cat local.conf
.....
# Set IMAGETEST to qemu if you want to build testcases and start
# testing in qemu after do_rootfs.
IMAGETEST = "qemu"
.....
  • Then you can run "bitbake poky-image-xxx" (xxx can be minimal, sato, sdk). After the kernel/image are generated, you need run "bitbake poky-image-xxx -c qemuimagetest". It will trigger automation test and you will get the result after a while. :)

Full Pass Specification

M2 Test Execution Plan

Since there are still some unclear requirements for Yocto project, currently it’s hard to define test plan to cover all future milestone/weekly testing. According to the suggestion from KangKang, I defined a test execution plan for M2 testing, which only covers tests for M2 requirement.


The fullpass test plan is here, M2 fullpass test plan. For each target, like qemux86, qemuarm, netbook, e-Menlow, etc, I define separate test plan for them, with sub-tests chosen from the fullpass test plan.

I setup a testlink page to track test case and test plan status, http://otc-qa.sh.intel.com/testlink_yocto/. You can use username/passwd: guest/guest, to browser test case and test plan.

Kernel

Distro

Yocto Release Process

Steps to release Yocto:

Run an autobuilder "generate-release" build. This will create images in /srv/www/vhosts/autobuilder.pokylinux.org/generate-release/<datestamp>/. Rename these files with the appropriate release name and version (Scott will have a script to do this by the next release).

For post-Green releases, there will be a release staging area that will be used to mirror the output before the files are released on the public web site.

On release branch:

  • Set DISTRO_VERSION in poky.conf
  • Update handbook references to stable release (introduction.xml, master branch needs this too)
  • Update version reference and updated date in handbook (poky-handbook.xml)
  • git commit -a -m "Purple 3.2 Release"
  • git tag -a purple-3.2 -m "Tag Purple 3.2"
  • git archive purple-3.2 --prefix=purple-3.2/ | bzip2 > /tmp/poky-purple-3.2.tar.bz2 scp /tmp/poky-purple-3.2.tar.bz2 o- hand.com:/srv/www/pokylinux.org/releases/poky-purple-3.2.tar.bz2

On master branch:

  • Set DISTRO_VERSION in poky.conf to new version
  • Update version reference and updated date in handbook (poky-handbook.xml)
  • cd handbook
  • make
  • scp -r poky-handbook.tgz poky-handbook.html poky-handbook.pdf *.png *.xml *.css *.svg o-hand.com:/srv/www/pokylinux.org/releases/purple-3.2/doc/
  • scp -r poky-handbook.tgz poky-handbook.html *.png *.xml *.css *.svg o-hand.com:/srv/www/pokylinux.org/releases/doc/

(or copy across by hand when remotely connected to machine?)

  • Edit web pages (website is controlled from a git repo)
    • support/index.php
    • getit/index.php
    • index.php
    • poky-wp-theme/common-funcs.inc
    • poky-wp-theme/front-page.php
  • Post release announcement on Blog
  • Post release announcement on mailing list

Core

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Processes_and_Activities&oldid=83"