Processes and Activities

From Yocto Project
Revision as of 20:20, 22 October 2010 by Scottrif (talk | contribs)
Jump to navigationJump to search

Welcome to the Processes and Activities Page!


QA

Milestone Test Report for Yocto

Fullpass Test Report for M3-20100911

Fullpass Test Report for M3-20100911

Sanity Test Report for M3-20100911

Sanity Test Report for M3-20100911

Sanity Test Report for M3-20100909

Sanity Test Report for M3-20100911

Partial Sanity Test Report for M3 20100905

Partial Test Report for M3 20100905

Fullpass Test for M2-20100731

Fullpass Test Report for M2-20100731

M2-20100725 & Bug Update for M2-20100728

I finished testing on QEMU/Netbook for M2-20100725 and verified some bugs with M2-20100728 build. Against M2-20100725 build, I reported 12 new bugs and verified 2 bugs as fixed. With the second build M2-20100728, I verified old critical bugs and reported another 2 new bugs. 2 old bugs are verified as fixed with this build.

M3 Test Execution Plan

M3 fullpass test plan.

M4 Test Execution Plan

M4 fullpass test plan.


There are still 5 open bugs for M2, 4 of them are CRITICAL. And 23 open bugs are marked to M3. As a summary, QEMU targets (x86/arm/mips/ppc) can work without critical bugs. Toolchain has one critical bug. Meanwhile, emenlow/netbook have critical issue with/within X environment.


Open Bugs for M2:

Other Open Bugs reported, target for M3 or later:

Verified Fixed Issues:

Existing Old Issues:

M1 Test Report 2010/7/10

Summary & Issue List

I finished sanitytest for M1 20100710 build. There is 2 new issues found in this testing. qemuppc boot up issue is fixed but qemu will report OOM error when booting qemuppc images. There is no mouse action response after X startup with qemumips images. qemuppc and qemuarm cannot shutdown entirely. There are totally 4 open bugs in this testing.

BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).

Issue List:

Test Result Matrix:

SanityTest Result

Machine Image Boot Network(ping) X startup SSH Service Shutdown Comments
qemux86 poky-image-sato PASS PASS PASS PASS PASS
poky-image-sdk PASS PASS PASS PASS PASS
qemuxarm poky-image-sato PASS PASS PASS PASS FAIL
poky-image-sdk PASS PASS PASS PASS FAIL
qemumips poky-image-sato PASS PASS PASS PASS PASS No mouse action response, some icons broken
poky-image-sdk PASS PASS PASS PASS PASS No mouse action response, some icons broken
qemuppc poky-image-sato PASS PASS N/A PASS FAIL
poky-image-sdk PASS PASS N/A PASS FAIL

Detail Test Result

Test Result can be found here Milestone Test Result for 20100710

Testing Env

Host: Ubuntu 10.04 LTS
Kernel: 2.6.32-21-generic-pae
Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)
Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100710-1/


M1 Test Report 2010/7/8

Summary & Issue List

I finished sanitytest for M1 20100708 build. There are 3 issues found in this testing. The major issue is that there is no cursor in qemuarm and qemumips images. I also tried qemuppc because there is some fixing in latest tree, but qemuppc still fails to startup. 5 issues found in last time testing are fixed. Qemuarm can boot up now. Qemuarm and qemumips can start X now.

BTW, This time I still build qemu-native from WR's branch (git.pokylinux.org:poky-wr +yocto-kernel-clean).

Issue List:

Test Result Matrix:

SanityTest Result

Machine Image Boot Network(ping) X startup SSH Service Shutdown Comments
qemux86 poky-image-sato PASS PASS PASS PASS PASS
poky-image-sdk PASS PASS PASS PASS PASS
qemuxarm poky-image-sato PASS PASS PASS PASS FAIL No cursor after X startup
poky-image-sdk PASS PASS PASS PASS FAIL No cursor after X startup
qemumips poky-image-sato PASS PASS PASS PASS PASS No cursor after X startup
poky-image-sdk PASS PASS PASS PASS PASS No cursor after X startup

Detail Test Result

Test Result can be found here Milestone Test Result for 20100708

Testing Env

Host: Ubuntu 10.04 LTS
Kernel: 2.6.32-21-generic-pae
Poky gitinfo(used to build qemu-native): aa416b6133bd55b5ba4d2fb6f5faef274ba511a0 (git.pokylinux.org:poky-wr +yocto-kernel-clean)
Build Image Download URL: http://autobuilder.pokylinux.org/milestone/20100708-1/

M1 Test Report 2010/6/29

Summary & Issue List

There are 8 target images be tested, only 3 ones can boot up, while SDK image for qemumips is not built out. Testing for qemuarm and qemuppc is blocked. Network in qemux86 image is not enabled. X failure is still within SDK image. And there is no cursor within qemux86 poky-image-sato. Detail issue list as below.

For BAT testing, LTP and POSIX testing are performed against qemux86 poky-image-sdk and qemumips poky-image-sato. Compared with last time test result, LTP result for qemux86 is same, while POSIX result has one more failure.

BTW, I build qemu-native by myself from poky WR’s branch (git.pokylinux.org:poky-wr). These qemu startup scripts, like “runqemu”, also come from the branch.

Issue List:

  1. No network in qemux86 image, (sato/sdk). It should be caused by lack of e1000 module built in WR kernel.
  2. No cursor after X startup in qemux86 poky-image-sato image.
  3. Qemux86 poky-image-sdk image fails to start X. (Known issue as Saul pointed)
  4. Fail to boot up qemuarm images, (sato/sdk). It seems no root partition when booting. See attachment “qemuarm-poky-image.log” for full log when booting QEMU. qemuarm log
  5. Fail to boot up qemuppc images, (sato/sdk). qemu reports “could not load PPC PREP bios ‘powerpc_rom.bin’”. See attachment “qemuppc-poky-image.log” for full log. qemuppc log
  6. By default, script “runqemu” does not start X for qemuarm, qemuppc and qemumips. I find there is no wnidow popup even I remove the X disable option, "-nographic" for qemumips.

Test Result Matrix:

SanityTest Result

Machine Image Boot Network(ping) X startup SSH Service Shutdown Comments
qemux86 poky-image-sato PASS FAIL PASS N/T PASS No cursor after X startup with show-cursor option
poky-image-sdk PASS FAIL FAIL N/T PASS
qemuxarm poky-image-sato FAIL BLOCK BLOCK BLOCK BLOCK
poky-image-sdk FAIL BLOCK BLOCK BLOCK BLOCK
qemumips poky-image-sato PASS PASS N/A PASS PASS option "-nographic" is set in script "poky-qemu-internal" for qemumips. There is no window popup even I remove the option.
poky-image-sdk N/A N/A N/A N/A N/A No SDK images for qemumips
qemuppc poky-image-sato FAIL BLOCK BLOCK BLOCK BLOCK
poky-image-sdk FAIL BLOCK BLOCK BLOCK BLOCK

BAT Result:

Machine Image LTP(FAIL/TOTAL) POSIX(FAIL/UNSUPPORT/TOTAL)
qemux86 poky-image-sdk 46/1120 51/99/1644
qemumips poky-image-sato 58/1116 N/A

Detail Test Result

Test Result can be found here Milestone Test Result for 20100629

Testing Env

Host: Ubuntu 10.04 LTS
Kernel: 2.6.32-21-generic-pae
Poky gitinfo(used to build qemu-native): 5280cfe8c9cd9c9a880683186c77c24d380978ed (git.pokylinux.org:poky-wr)
Build Image Download URL: Except for SDK images for qemuppc and qemuarm (from http://autobuilder.pokylinux.org/nightly/20100629-1/), all the other images come from http://autobuilder.pokylinux.org/nightly/20100628-1/.

Enable Automation Test in Poky

Currently we have implemented two automated sanity test cases in Poky. User can follow below steps to enable the automation test:

  • Make sure your working tree has latest test code. The first sanity test implementation is checked in poky master tree as commit, fd1d661b391dfad1edaac937a17c1165f635031a.
  • Make sure your machine has package "expect" installed. The package is necessary for testing. We will add a recipe for it later. Currently, you need install it by yourself. On Ubuntu host, you can use apt-get to install expect:
$ apt-get install expect
  • Edit /etc/sudoers to allow the user run sudo without prompted for a password. For example, you can do like following for user "tester":
$ cat /etc/sudoers
# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the man page for details on how to write a sudoers file.
#
....
....
....

%tester ALL=(ALL) NOPASSWD: NOPASSWD: ALL
  • Typically, you need a X environment to start QEMU. You can use your desktop's DISPLAY. Or, alternatively, you can install vncserver, and startup a X environment in localhost:x.0. (x can be 1, 2, 3.....n)
  • You need set "IMAGETEST" to "qemu" in build/conf/local.conf
$ cat local.conf
.....
# Set IMAGETEST to qemu if you want to build testcases and start
# testing in qemu after do_rootfs.
IMAGETEST = "qemu"
.....
  • Then you can run "bitbake poky-image-xxx" (xxx can be minimal, sato, sdk). After the kernel/image are generated, you need run "bitbake poky-image-xxx -c qemuimagetest". It will trigger automation test and you will get the result after a while. :)

Full Pass Specification

M2 Test Execution Plan

Since there are still some unclear requirements for Yocto project, currently it’s hard to define test plan to cover all future milestone/weekly testing. According to the suggestion from KangKang, I defined a test execution plan for M2 testing, which only covers tests for M2 requirement.


The fullpass test plan is here, M2 fullpass test plan. For each target, like qemux86, qemuarm, netbook, e-Menlow, etc, I define separate test plan for them, with sub-tests chosen from the fullpass test plan.

I setup a testlink page to track test case and test plan status, http://otc-qa.sh.intel.com/testlink_yocto/. You can use username/passwd: guest/guest, to browser test case and test plan.

Kernel

Distro

Yocto Release Process

Steps to release Yocto:

Run an autobuilder "generate-release" build. This will create images in /srv/www/vhosts/autobuilder.pokylinux.org/generate-release/<datestamp>/. Rename these files with the appropriate release name and version (Scott will have a script to do this by the next release).

For post-Green releases, there will be a release staging area that will be used to mirror the output before the files are released on the public web site.

On release branch:

  • Set DISTRO_VERSION in poky.conf
  • Update handbook references to stable release (introduction.xml, master branch needs this too)
  • Update version reference and updated date in handbook (poky-handbook.xml)
  • git commit -a -m "Purple 3.2 Release"
  • git tag -a purple-3.2 -m "Tag Purple 3.2"
  • git archive purple-3.2 --prefix=purple-3.2/ | bzip2 > /tmp/poky-purple-3.2.tar.bz2 scp /tmp/poky-purple-3.2.tar.bz2 o- hand.com:/srv/www/pokylinux.org/releases/poky-purple-3.2.tar.bz2

On master branch:

  • Set DISTRO_VERSION in poky.conf to new version
  • Update version reference and updated date in handbook (poky-handbook.xml)
  • cd handbook
  • make
  • scp -r poky-handbook.tgz poky-handbook.html poky-handbook.pdf *.png *.xml *.css *.svg o-hand.com:/srv/www/pokylinux.org/releases/purple-3.2/doc/
  • scp -r poky-handbook.tgz poky-handbook.html *.png *.xml *.css *.svg o-hand.com:/srv/www/pokylinux.org/releases/doc/

(or copy across by hand when remotely connected to machine?)

  • Edit web pages (website is controlled from a git repo)
    • support/index.php
    • getit/index.php
    • index.php
    • poky-wp-theme/common-funcs.inc
    • poky-wp-theme/front-page.php
  • Post release announcement on Blog
  • Post release announcement on mailing list

Core

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Processes_and_Activities&oldid=78"