QA: Difference between revisions
Line 40: | Line 40: | ||
Yocto Project QA heavily relies in the Autobuilder thanks to the aforementioned scheduled and on demand test execution features. | Yocto Project QA heavily relies in the Autobuilder thanks to the aforementioned scheduled and on demand test execution features. | ||
===Image Testing=== | |||
In order to execute tests in an image, it is necessary to boot it in either a virtual or a physical target. The execution in a virtual environment has a nice flow, documented in the [[Image_tests#Enabling_and_running_the_tests|Image Tests Enabling...]] section. And for executing tests in physical targets it would be required to: | |||
===Adding | # Boot the image in the target by following the [http://www.yoctoproject.org/docs/2.2/mega-manual/mega-manual.html#building-an-image-for-hardware building an image for hardware] instructions. | ||
# Run the image tests by following the [http://www.yoctoproject.org/docs/2.2/mega-manual/mega-manual.html#exporting-tests exporting tests] instructions. | |||
====Test Target Configuration==== | |||
A Device Under Test (DUT) | |||
===Creating and Adding New Tests=== | |||
Tests for a given component can be automated in the AutoBuilder. With that purpose, follow the [[AutoBuilder/Adding Automated Tests|Adding Automated Tests]] to the Autobuilder Guide. | Tests for a given component can be automated in the AutoBuilder. With that purpose, follow the [[AutoBuilder/Adding Automated Tests|Adding Automated Tests]] to the Autobuilder Guide. |
Revision as of 20:37, 2 November 2016
Current Release QA Trackers
- Bugs that need to be implemented by QA Team
- Bugs that need to be verified by QA Team
- Features to verify
- Features to implement
- Old Bugs
Current Release QA Test Plans
- QA Master Test Plan
- Yocto Project 2.2_Release Test Plan
- 2.2 QA Status
- Performance Charts
- Toaster testing plan
- Extensible SDK Test Plan (eSDK)
- Distro Testing Plan
- BSP Test Plan
Current Release QA Test Cases
Test Execution
Autobuilder
The AutoBuilder is Yocto Project's tool for non-manual test execution, it performs the following functions:
- A scheduled nightly build and test execution that includes:
- That each image created executes the corresponding set of image/run-time tests
- Specific Autobuilder tasks for running build-time testing
- A service for on demand testing requests. (Partially working, feature in progress at request #9880)
Yocto Project QA heavily relies in the Autobuilder thanks to the aforementioned scheduled and on demand test execution features.
Image Testing
In order to execute tests in an image, it is necessary to boot it in either a virtual or a physical target. The execution in a virtual environment has a nice flow, documented in the Image Tests Enabling... section. And for executing tests in physical targets it would be required to:
- Boot the image in the target by following the building an image for hardware instructions.
- Run the image tests by following the exporting tests instructions.
Test Target Configuration
A Device Under Test (DUT)
Creating and Adding New Tests
Tests for a given component can be automated in the AutoBuilder. With that purpose, follow the Adding Automated Tests to the Autobuilder Guide.
A list of tests that are automated can be seen here.
Reporting
- Bug reporting and Information levels
- Testopia, the test manager
- The Test Reporting Tool
- error report web
Performance testing
QA Resources
Archive
You can find the previous QA work by release in the Yocto Project QA Archive.
Other Relevant Data
- Yocto Bug Trend
- Compliance Test Result
- ADT Testing
- Regression Test
- Performance Test
- Distro Test
- Distribution Support
- QA BKM sharing
- LAVA server vs Yocto HW automation testing
- Note: The LAVA framework usage stopped in favor of testing in the AutoBuilder in early 2016.
List of Automated Tests
- Distribution Support
- add ptest wiki
- add piglit test wiki
- add kernel test wiki
- LSB
- LSB Result
- LTP
- LTP result
- POSIX
- Posix result
- POSIX-results
- POSIX History Results
- Automated package upgrade testing