QA: Difference between revisions

From Yocto Project
Jump to navigationJump to search
(Undo revision 53979 by Yeoh, Ee Peng (talk))
(Undo revision 53973 by Sangeeta Jain (talk))
Line 10: Line 10:


*QA Execution History
*QA Execution History
**[[Yocto_Project_2.7_Release_Test_Plan#Live_Schedule_.26_Execution_History | 2.7 qa run history]] - no need
**[[Yocto_Project_2.7_Release_Test_Plan#Live_Schedule_.26_Execution_History | 2.7 qa run history]]


*Bugs that need to be implemented by QA Team
*Bugs that need to be implemented by QA Team
**[[2.7_QA_OWNED_BUGS | 2.7 QA Assigned Bugs]] - no need
**[[2.7_QA_OWNED_BUGS | 2.7 QA Assigned Bugs]]


*Bugs that need to be verified by QA Team  
*Bugs that need to be verified by QA Team  
**[[2.7_QA_Bugs_To_Verify]] - no need
**[[2.7_QA_Bugs_To_Verify]]
<!--
<!--
*Features to verify  
*Features to verify  
**[[2.5 qa_owned features to verify]] - no need
**[[2.5 qa_owned features to verify]]


*Features to implement  
*Features to implement  
**[[2.5 qa owned features]] - no need
**[[2.5 qa owned features]]
-->
-->


*Old Bugs that need to be Verified
*Old Bugs that need to be Verified
**[[Old resolved bugs and features]] - no need
**[[Old resolved bugs and features]]


*2.7 Test Run Templates
*2.7 Test Run Templates
**[[Yocto_Project_2.7_Release_Test_Plan#Test_Items | Test Run Templates]] - no need
**[[Yocto_Project_2.7_Release_Test_Plan#Test_Items | Test Run Templates]]


==Current Release QA Test Plans==
==Current Release QA Test Plans==
* [[QA Master Test Plan]] -- keep the link; update page
* [[QA Master Test Plan]]
* [[Yocto Project 2.7_Release Test Plan]] -- TBD
* [[Yocto Project 2.7_Release Test Plan]]
* [[2.7 QA Status]] - no need
* [[2.7 QA Status]]
* [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html  Performance Charts ] -- TBD
* [https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html  Performance Charts ]
* [[Toaster testing plan]] -- TBD
* [[Toaster testing plan]]
* [[Extensible SDK Test Plan (eSDK)]] -- TBD
* [[Extensible SDK Test Plan (eSDK)]]
* [[Distro Testing Plan]] -- TBD
* [[Distro Testing Plan]]
* [[BSP Test Plan]] -- TBD
* [[BSP Test Plan]]


==Current Release QA Test Cases==
==Current Release QA Test Cases==
* [[Yocto 2.7 Test Cases]] - no need
* [[Yocto 2.7 Test Cases]]


 
==Test Execution==
==Test Execution== -- TBD
===Autobuilder===
===Autobuilder===


Line 83: Line 82:


==Reporting==
==Reporting==
*[[Bug reporting and Information levels]] -- keep but review
*[[Bug reporting and Information levels]]
*[[Testopia]], the test manager -- no need
*[[Testopia]], the test manager
*[[QA/Test_Reporting_Tool|The Test Reporting Tool]] -- TBD
*[[QA/Test_Reporting_Tool|The Test Reporting Tool]]
*[http://errors.yoctoproject.org/Errors/ error report web] --TBD
*[http://errors.yoctoproject.org/Errors/ error report web]
*[[QA REPORT TEMPLATE | Wiki of QA Report Template]] -- no need
*[[QA REPORT TEMPLATE | Wiki of QA Report Template]]


==Performance testing==
==Performance testing==
[[Performance Test]] -- TBD
[[Performance Test]]


==QA Resources==
==QA Resources==
*[[Rpm's Repository Setup for QA]] -- TBD
*[[Rpm's Repository Setup for QA]]
*[[Testopia]] -- TBD
*[[Testopia]]
*[[Testing Cycle]] -- TBD
*[[Testing Cycle]]
*[[qa-tools|qa-tools Git Repository]] -- TBD
*[[qa-tools|qa-tools Git Repository]]


= Archive =
= Archive =

Revision as of 03:19, 18 June 2019

New QA Process

Current Release QA Trackers

Current Release QA Test Plans

Current Release QA Test Cases

Test Execution

Autobuilder

The AutoBuilder is Yocto Project's tool for non-manual test execution, it performs the following functions:

  • A scheduled nightly build and test execution that includes:
  • A service for on demand testing requests. (Partially working, feature in progress at request #9880)

Yocto Project QA heavily relies in the Autobuilder thanks to the aforementioned scheduled and on demand test execution features.

Image Testing

In order to execute tests in an image, it is necessary to boot it in either a virtual or a physical target.

Testing Images in Virtual Targets:

The execution in a virtual environment has a nice flow, documented in the Image Tests Enabling... section.

Testing Images in Physical Targets:

For executing tests in physical targets it would be required to:

  1. Boot the image in the target by following the building an image for hardware instructions.
  2. Run the image tests by following the image test or exporting tests instructions.

Setting up Targets with Devauto

Manual instructions for setting up the physical test targets appear in many parts of the Yocto Project documentation (i.e. here). It is easy to setup one target using those instructions but it becomes challenging for the cases where multiple targets have to be prepared or the case where it is required to serialize a changing setup over time, for one: testing on several images using the same target.

Devauto is the Python library and command line interface intended to manage the device automation assets that act upon the target's physical state. Refer to the Devauto documentation for more information about the hardware supported and the library and CLI functionality.

Creating and Adding New Tests

Tests for a given component can be automated in the AutoBuilder. With that purpose, follow the Adding Automated Tests to the Autobuilder Guide.

A list of tests that are automated can be seen here.

Reporting

Performance testing

Performance Test

QA Resources

Archive

You can find the previous QA work by release in the Yocto Project QA Archive.

Other Relevant Data

List of Automated Tests