Toaster testing plan

From Yocto Project
(Difference between revisions)
Jump to: navigation, search
(Functionality tests)
(Test Cycle)
Line 63: Line 63:
| qemux86-64  ||  ||  || Yes
| qemux86-64  ||  ||  || Yes
| rowspan="3" | Target image|| core-image-minimal|| Yes || Yes ||  
| rowspan="3" | Target image|| core-image-minimal|| Yes || ||  
| core-image-sato-sdk|| || Yes || Yes
| core-image-sato-sdk|| || Yes || Yes

Revision as of 07:03, 29 November 2013



This article is the test plan for Toaster.


The test process focuses on:

  • validating the data collected from the build process;
  • verifying the correct functioning of the Toaster GUI; TBD

Test Areas

Toaster consists of two big components, as follows:


Functionality tests

  • REST API verification – create Django tests to detect API calls returning incomplete data (fields having null values) - our goal is to have 100% data collected;
  • Calculation of the data collection rate - the ratio between the number of the variables having null values ​​and the total number of variables collected (the data collection rate does not include wrong values);
  • Verify that all links in the simple UI are available;
  • Verify the quality of the data collected through the simple UI;
  • Verify the quality of the data collected by verifying most of the values from the database by connecting directly to the database - our immediate goal is to have 90% correct data collected;

Usability tests


Functionality tests

  • Manual testing in the first stage;
  • Automate testing using Selenium, in the second stage;

Compatibility tests

  • Verify the behavior of the GUI on different browsers and operating systems; TBD

Usability tests

Performance tests

  • Stress testing (e.g. display appropriate error messages when the system is under stress);

Test Cycle

Test execution cycle
#Weekly Test #Full Pass Test #Release Test
Build type Weekly Yes Yes
Release Yes Yes Yes
#Test Areas #Backend Yes Yes Yes
#Frontend Yes Yes Yes
Target machine qemuarm Yes Yes
qemumips Yes
qemuppc Yes
qemux86 Yes Yes Yes
qemux86-64 Yes
Target image core-image-minimal Yes
core-image-sato-sdk Yes Yes

Weekly Test

  • Scope
    • Images built weekly and released through the distribution team.
  • Objective
    • Functionality test on most areas with minimum sets of tests;
    • Regression test with high probability to find bugs.

Full Pass Test

  • Scope
    • Images built as candidates for milestone or final release;
    • Passed #Weekly Test
  • Objective
    • Ensure functionality of Toaster component.

Release Test

  • Objective
    • All scheduled features are covered, or rescheduled;
    • All relevant bugs are fixed and verified.
Personal tools