Yocto Project v1.0 Release Criteria: Difference between revisions

From Yocto Project
Jump to navigationJump to search
(added March 29 status)
 
(25 intermediate revisions by the same user not shown)
Line 7: Line 7:
|-
|-
! scope="row" | Functional Completion
! scope="row" | Functional Completion
| All M1, M2, M3 functional features implemented by EO M3
| ''M4 (Gold) Criteria:  All M1, M2, M3 features and BSPs are in build.''
''M4 (Gold) Criteria:  All features and BSPs are in build.''
| bgcolor="green" | see [[Yocto 1.0 Schedule#Stabilization Phase Features and Activities]]
| bgcolor="yellow" | see [[Yocto 1.0 Schedule#Stabilization Phase Features and Activities]]
All planned features are in build.
Richard will send a note regarding the four open features.
|-
|-
! scope="row" | Build and Release
! scope="row" | Build and Release
(Yocto)
(Yocto)
| Yocto 1.0 code builds reliably
|''M4 (Gold) criteria:  All features in master can be built with no errors.''
''M4 (Gold) criteria:  All features in master can be built with no errors.''
| bgcolor="green" | Builds are green for RC4 and RC5.<br>
| bgcolor="orange" | Four issues:
RC4:  DISTRO_VERSION isn't at 1.0.<br>
- We had an issue with builder02 running out of disk space, so some of the build copying was done post-build. This did not effect the build and was dealt with manually after cleaning up the partition.
RC5: DISTRO_VERSION is at 1.0.
 
- There is an issue with how package-index is being run. I will need to sync up with Jessica to verify how this should be incorporated into the build.
 
- The sanity tests on builder02 failed because builder02 was unable to resolve "localhost". This has been corrected and the sanity tests re-run incrementally: http://autobuilder.pokylinux.org:8010/builders/nightly-internal-sanity/builds/3
 
- The tarball generation buildstep is not branch aware. This will be fixed this week.  In the meantime it was run manually.
|-
|-
! scope="row" | Build and Release
! scope="row" | Build and Release
(BSPs)
(BSPs)
| Yocto BSPs build reliably
| ''M4 (Gold) criteria:  Core and meta-intel BSPs can be built with no errors.''
''M4 (Gold) criteria:  BSPs can be build from master with no errors.''
| bgcolor="green" | Builds are green.
| bgcolor="yellow" |  
- BSPs did not complete successfully. There were some issue in the actual build mostly centering around:
-- Failing to do_kernel_checkout
-- Unpack issues around svn://svn.o-hand.com/repos
 
- Beagleboard failed doing runstrip on uboot during do_populate_sysroot
|-
|-
! scope="row" | Open Bugs
! scope="row" | Open Bugs
| All bugs dispositioned; Show-stoppers fixed
| ''M4 (Gold) criteria:  All bugs have been dispositioned.  Critical/High bugs targeted for M4 have been fixed.''
''M4 (Gold) criteria:  All bugs have been dispositioned.  Bugs targeted for M4 have been fixed.''
| bgcolor="green" | RC4 and RC5:  All High priority bugs have been dispositioned.  All High priority bugs for 1.0 are fixed. *
| bgcolor="orange" | 76 total open defects targeted for 1.0
Critical: 0
High: 14
Medium: 53
Low: 7
Undetermined: 2
|-
|-
! scope="row" | Functional Tests
! scope="row" | Functional Tests
| 100% of planned tests executed and all bugs are logged in bugzilla
| ''M4 (Gold) criteria:  100% of planned tests are executed and results are logged in bugzilla''
''M4 (Gold) criteria:  All planned tests are executed and results are logged in bugzilla''
| bgcolor="green" |
| bgcolor="yellow" | - zypper-search failed on qemux-x86_64 on poky-image-lsb and poky-image-sato
RC4:  QA has completed test pass on RC4 except for stress testing. * <br>
RC5:  QA has completed sanity test for RC5.
|-
|-
! scope="row" | BSP Testing
! scope="row" | BSP Testing
| 100% of planned BSP tests executed
| ''M4 (Gold) criteria:  100% of planned BSP tests are executed''
''M4 (Gold) criteria:  All planned BSP tests are executed''
| bgcolor="green" |  
| bgcolor="orange" | Team hasn't get been able to do a full BSP test pass.  Open issues exist with respect to the BSPs (Julie).
RC4:  QA has completed test pass on RC4 except for stress testing. * <br>
RC5: QA has completed sanity test for RC5.
|-
|-
! scope="row" | Alpha Feedback
! scope="row" | Alpha Feedback
| Alpha feedback included
| ''M4 (Gold) criteria:  Alpha runs Feb 11th - Feb 23rd.  Feedback is dispositioned.''
''M4 (Gold) criteria:  Alpha begins Feb 11th and completes Feb 23rd''
| bgcolor="#66FF33" | Program completed.  Issues have been dispositioned.
| bgcolor="#66FF33" | Program completed.  Someone just needs to parse final results.
|-
|-
! scope="row" | Documentation
! scope="row" | Documentation
| Release documentation created and published
| ''M4 (Gold) criteria:  Release documentation created and published''
''M4 (Gold) criteria:  M4 schedule items completed''
| bgcolor="green" | Planned documentation deliverables are completed.  See [[Yocto_1.0_Schedule#Documentation_Deliverables]]
| bgcolor="#66FF33" | [[Yocto_1.0_Schedule#Documentation_Deliverables]]
|-
|-
! scope="row" | Build Performance  
! scope="row" | Build Performance  
| Build takes 90 minutes (stretch goal = one hour) to do a complete rebuild of local sources of poky-image-sato on a developer's desktop.  (The one we are using is: Intel(R) Core(TM) i7 CPU 940 @ 2.93GHz, 8 siblings, 4 cpu cores, total mem 6118644k, with: MACHINE=qemux86, BB_NUMBER_THREADS = "8", PARALLEL_MAKE = "-j 6", and only rpm packaging.)
| ''M4 (Gold) criteria:  Build takes 90 minutes (stretch goal = one hour) to do a complete rebuild of local sources of poky-image-sato on a developer's desktop.  (The one we are using is: Intel(R) Core(TM) i7 CPU 940 @ 2.93GHz, 8 siblings, 4 cpu cores, total mem 6118644k, with: MACHINE=qemux86, BB_NUMBER_THREADS = "8", PARALLEL_MAKE = "-j 6", and only rpm packaging.)''
 
| bgcolor="#66FF33" | At 105 minutes.  Team decision that this is not a blocker for 1.0.
''M4 (Gold) criteria:  Goal above is reached.''
|-
| bgcolor="yellow" | Team is down to 97-98 minute builds. Richard will continue the analysis/tweak cycle; however, attention needs to go to other areas as well this week. File dependencies is one other low-hanging fruit item Richard would like to investigate.
! scope="row" | ADT User Workshop
real 109m52.400s
| ''M4 (Gold) criteria:  ADT User Workshop is held where developers use ADT tools to demo software, and results are included.''
user 393m58.840s
| bgcolor="#66FF33" | Running into problems recruiting people, but not a blocker for 1.0 release.
sys 59m25.380s
|-
 
! scope="row" | Upstream Commits
| ''M4 (Gold) criteria:  No goals for Yocto 1.0 - Yocto 1.1 will include more goals.''
||
|-
|-
! scope="row" | Marketing requirements complete
! scope="row" | Marketing requirements from Engineering complete
| Engineering requirements to the marketing plan are complete
| ''M4 (Gold) criteria:  Yocto demo is prepared for ELC.''
''M4 (Gold) criteria:  Yocto demo is prepared for ELC.''
| bgcolor="yellow" | Top issues:<br>
| bgcolor="yellow" | Hardware plan defined. Need videos.  Need plexiglass boxes.  Coding effort underway, but not complete.
Need to fix bug 919 so that rygel works and audio/video can be played.- Josh had success with previous rygel version.<br>
Need to get video to work on at least one piece of hardware.  (Tom working on.)<br>
Need to ensure PPC can be used as a Content Store.  (Scott G is working on once builds are done.) <br>
|-
|-
! scope="row" | Publication
! scope="row" | Publication
| Release and documentation are published on yoctoproject.org
| ''M4 (Gold) criteria:  Release and documentation are published on yoctoproject.org and Yocto 1.0 is announced''
''M4 (Gold) criteria:  Yocto Project 1.0 published and announced''
| bgcolor="yellow" | Release process beginning.  See [[Yocto 1.0 Release Checklist]] for details.
| bgcolor="white" |  
|}
|}


Line 101: Line 85:
! scope="row" | Build and Release
! scope="row" | Build and Release
| EMGD and non-EMGD BSPs build reliably
| EMGD and non-EMGD BSPs build reliably
| bgcolor="#66FF33" | Build completed
| bgcolor="green" | Build completed
|-
|-
! scope="row" | Open Bugs
! scope="row" | Open Bugs
Line 109: Line 93:
! scope="row" | Functional Tests
! scope="row" | Functional Tests
| 100% of planned tests executed
| 100% of planned tests executed
| bgcolor="#66FF33" | Tom tested.  Jiajun will test Thursday.
| bgcolor="green" | 100% testing complete
|-
|-
! scope="row" | Legal Compliance
! scope="row" | Legal Compliance
| All legal requirements met
| All legal requirements met
| bgcolor="yellow" | Account is set up for click-through.  Click-through team has info they need.
| bgcolor="green" | Click-through completed
|-
|-
! scope="row" | Documentation
! scope="row" | Documentation
| Release Notes created and published
| Release Notes created and published
| bgcolor="#66FF33" | Scott/Tom are finalizing and things look good.
| bgcolor="green" | Done
|-
|-
! scope="row" | Performance  
! scope="row" | Performance  
Line 125: Line 109:
! scope="row" | Publication
! scope="row" | Publication
| Release and documentation are published on yoctoproject.org
| Release and documentation are published on yoctoproject.org
| bgcolor="yellow" | on-track
| bgcolor="green" | Done
|}
|}


Latest revision as of 16:02, 29 March 2011

Yocto v1.0 Release Criteria

Release Criteria
Release Criteria Target Status
Functional Completion M4 (Gold) Criteria: All M1, M2, M3 features and BSPs are in build. see Yocto 1.0 Schedule#Stabilization Phase Features and Activities

All planned features are in build.

Build and Release

(Yocto)

M4 (Gold) criteria: All features in master can be built with no errors. Builds are green for RC4 and RC5.

RC4: DISTRO_VERSION isn't at 1.0.
RC5: DISTRO_VERSION is at 1.0.

Build and Release

(BSPs)

M4 (Gold) criteria: Core and meta-intel BSPs can be built with no errors. Builds are green.
Open Bugs M4 (Gold) criteria: All bugs have been dispositioned. Critical/High bugs targeted for M4 have been fixed. RC4 and RC5: All High priority bugs have been dispositioned. All High priority bugs for 1.0 are fixed. *
Functional Tests M4 (Gold) criteria: 100% of planned tests are executed and results are logged in bugzilla

RC4: QA has completed test pass on RC4 except for stress testing. *
RC5: QA has completed sanity test for RC5.

BSP Testing M4 (Gold) criteria: 100% of planned BSP tests are executed

RC4: QA has completed test pass on RC4 except for stress testing. *
RC5: QA has completed sanity test for RC5.

Alpha Feedback M4 (Gold) criteria: Alpha runs Feb 11th - Feb 23rd. Feedback is dispositioned. Program completed. Issues have been dispositioned.
Documentation M4 (Gold) criteria: Release documentation created and published Planned documentation deliverables are completed. See Yocto_1.0_Schedule#Documentation_Deliverables
Build Performance M4 (Gold) criteria: Build takes 90 minutes (stretch goal = one hour) to do a complete rebuild of local sources of poky-image-sato on a developer's desktop. (The one we are using is: Intel(R) Core(TM) i7 CPU 940 @ 2.93GHz, 8 siblings, 4 cpu cores, total mem 6118644k, with: MACHINE=qemux86, BB_NUMBER_THREADS = "8", PARALLEL_MAKE = "-j 6", and only rpm packaging.) At 105 minutes. Team decision that this is not a blocker for 1.0.
ADT User Workshop M4 (Gold) criteria: ADT User Workshop is held where developers use ADT tools to demo software, and results are included. Running into problems recruiting people, but not a blocker for 1.0 release.
Upstream Commits M4 (Gold) criteria: No goals for Yocto 1.0 - Yocto 1.1 will include more goals.
Marketing requirements from Engineering complete M4 (Gold) criteria: Yocto demo is prepared for ELC. Top issues:

Need to fix bug 919 so that rygel works and audio/video can be played.- Josh had success with previous rygel version.
Need to get video to work on at least one piece of hardware. (Tom working on.)
Need to ensure PPC can be used as a Content Store. (Scott G is working on once builds are done.)

Publication M4 (Gold) criteria: Release and documentation are published on yoctoproject.org and Yocto 1.0 is announced Release process beginning. See Yocto 1.0 Release Checklist for details.

Tunnel Creek BSP Release Criteria

Release Criteria
Release Criteria Target Status
Development progress Features implemented; Code completed Coding completed
Build and Release EMGD and non-EMGD BSPs build reliably Build completed
Open Bugs All bugs dispositioned; Show-stoppers fixed Only minor bugs found (724, 725)
Functional Tests 100% of planned tests executed 100% testing complete
Legal Compliance All legal requirements met Click-through completed
Documentation Release Notes created and published Done
Performance Performance targets achieved This will be done out of cycle.
Publication Release and documentation are published on yoctoproject.org Done

Laverne Point Release Criteria

Release Criteria
Release Criteria Target Status
Development Progress Coding complete Coding completed
Build and Release Code builds reliably Release candidate is built.
Open Bugs All bugs dispositioned; Show-stoppers fixed No opens are show-stoppers. Open issues will be ID'd in release notes.
Functional Tests 100% of planned tests executed testing completed
Legal Compliance All legal requirements met tools run; requirements met
Performance Performance targets achieved N/A
Publication Release and documentation are published on yoctoproject.org Complete
Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Yocto_Project_v1.0_Release_Criteria&oldid=1051"