Toaster testing plan

This article is the test plan for Toaster.

Objectives

Verify all Toaster components to be fully functional.

Components to be verified:

• UI interface
• Backend interaction with bitbake for build purposes
• Backend interaction with database for storing and accessing build information

The testing objective involves only positive testing for existing features on Toaster. Perform exploratory testing focusing on newer features

• this can sometimes generate new test cases.

Verifying the bugs and features.

Scope

Testing scope has been agreed upon as local installation only, using sqlite database on Fedora and Ubuntu operating systems on Firefox browser.

New features are to be tested as they are finished with test cases written for them once features are stabilized.

UI features being tested:

• Create new project
• Modify project settings and variables
• Add/Remove layers to project
• Building images
• Build details
• All builds report table

Backend features:

• Toaster installation and first time configuration
• Data presence
• Data consistency
• Build data

 

Test Strategy

There are several test approaches for Toaster Testing, such as:

• Perform test cases agreed upon the development during periodic full passes according to the Schedule
• Write new test cases based on developer request or new features requires it.
  • Maintain current test cases and update them accordingly
• Perform exploratory testing on existing functionalities.
  • YP 2.1 Testopia template here.

Major feature groups and their testing approach

FrontEnd - UI Basic functionalities

Toaster UI features are tested such as links, buttons, tables, pages basic functionalities.

Testing needs :

• Built only in python 2.7
• Latest version of Selenium as python module
• Latest version of Firefox
• In order to test Selenium automated based tests (https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4027 ), the following steps need to be taken before running, on a clean toaster setup
  • ckeck the steps and requirements for “The Selenium automated UI tests” from https://wiki.yoctoproject.org/wiki/Testing_Toaster
• For UI manual testing follow the steps from https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4029 TCs
  • currently working on their automation using Selenium

BackEnd functionalities

Covers data integrity, data transfer between Bitbake and Toaster, Start/Stop, building images, configuration, etc

Testing needs:

• Built in python 2.7
• Direct sql requests to local sqlite instance
• Requires sqlite3 client installed (for manual debugging/investigation)
• BackEnd automated tests are verifying the correctness of the data gathered by the build process as presented in the Backend template: https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4028
  • In order for these tests to run properly, check the the steps for “The oe-selftest toaster tests” from https://wiki.yoctoproject.org/wiki/Testing_Toaster
• For BackEnd managed mode manual testing follow the steps from the following template https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4030
  • currently working on their automation
  • AR Cristina + Mihai:
• please validate and add info from Testing Toaster wiki in this section(Test Strategy): https://wiki.yoctoproject.org/wiki/Testing_Toaster

Test Approach

Overall approach to testing. Specify major activities, techniques and tools used to test groups of features

Deliverables

Deliverables

• Features to be tested

Dependencies

• Personal Dependencies
• Software Dependencies
• Hardware Dependencies