Qa-tools

Summary

The qa-tools git repository is the recipient for the QA automation scripts and tools. Is inteended to store any tool that helps on the QA activities related to the Yocto Porject

You can take a look at it via the gitweb interface.

For cloning the qa-tools repository:

$ git clone git://git.yoctoproject.org/qa-tools

List of Tools

Below is a list with detailed information about the tools/scripts already implemented

testopia_update

Objective

This tool was designed to give an automated way to interact with Testopia activities.

Structure

This tool allows to create a test run based on [ Templates ] already defined for every release, and based on that the tool is able to create the appropiate test run by cloning the exiting template and generating a new Test Run that will show the results of the test execution. Tool consist in a python script that can be used for different actions and with different parameters, with a configuration file to get connected to the Testopia DB and then using the Testopia API to perform different actions on the DB.

The structure is as follows:

-> qa-tools/ 
  |-testopia_update.py: Main script 
  |-testopia_update.config : Configuration file with the URL and the login info to the Testopia DB
  |-> testopia_update/
     |- __init__.py
     |-> product/ : This directory contains all the available products used by QA on Testopia                
  |-> External/
     |- testopia.py: This file contains the API that provides methods to work on Testopia DB.

How to use it

1- Clone repo

2- Install Virtual Environment tool

$ sudo apt-get install python-virtualenv

3- Create virtual environment

$ virtualenv venv

4- Activate virtual environment

$ source venv/bin/activate

5- Create configuration file, there is already a configuration file example that can be used

testopia_update.config.example [snapshot]

[Testopia]
url = TESTOPIA URL - https://bugzilla.yoctoproject.org/xmlrpc.cgi 
username = someuser - Bugzilla user
password = verysecurepass - Bugzilla pswd
store_location = /tmp/testopia_update

6- Install requirements

$ pip install -r requeriments.txt

7- Start using the testopia_update.py script

./testopia_update.py -h 

Below is a brief explanation of each one of the parameters available for the script:

-h, --help

Show this help message and exit

--config

Configuration file with the updated data to get connected to the Testopia DB

--url

URL of Testopia instance. This is used in case a configuration file is not being specified

--username

Username of Testopia instance. This is used in case a configuration file is not being specified

--password

Username of Testopia instance. This is used in case a configuration file is not being specified

--list-products

List available products. This option retrieves the list of the available products that can be used on Testopia instance

-a --action {create,update}

Action to execute can be create or update. This should be combined with other option check [example] section to see how can be used

-p --product

Product to create or update. This is the name of the product available on Testopia. This should be one of the valid products retrieved when using option --list-products

-c --category {AUTO,MANUAL}

Category for create or update. This represents the way how the Test Cases of the test run are being executed

-b --branch {master,jethro,dizzy,daisy,noexists}

Branch for create or update. Thi is the branch that is currently tested. Generally we used master branch as it represente the curren release under test

-e --environment

Environment for create or update.This is teh environemt where the test cases are being executed, could be a Distro OS, VM or real HW

-o --optional

Optional parameter for get Test runs. This is a string added to the name of the Test Run usually used on BSPs to specify the image where the test cases will be executed

--project-version

Version of the project. This is the release of the project that is being validated, to check current version go to [Releases]

--project-milestone

Milestone of the project. This representes the milestone of the project where Tes Run will be applicable, it should be added also the release (i.e 2.3 M1)

--project-revision

SCM Revision of the project. This is a combination of the Release candidate and the Hash of that release candidate on Poky go to [Examples] to check how this is used

--project-date PROJECT_DATE

SCM version/revision date of the project. The date where the release candidate outs at main auto builder, format is YYYY-MM-DD

--test-plan

Name of the test plan of the product, used when test plan name is different from product name. Normally the test plans on Testopia are composed as "<PRDOUCT_NAME>: <BRANCH_NAME> branch" but sometimes the test plan name is not the same as the product name so the correct test plan name should be specified.

--testrun-id

Number of the test run to be updated, this option should be used along with update action. If you already know the ID of the test run to be updated this option should be used to avoid all other parameters to search for the test run.

--results-log

Results log. This is the log generated by the execution were are reported the Test Cases IDs with the corresponding status (PASSED | FAILED)

--verbose

Enable verbose mode.

--debug

Enable debug mode.

Examples

Create a Test Run

./testopia_update.py 
                        --config testopia_update.config
                        -a create 
                        -p ADT 
                        -c AUTO 
                        -b master 
                        -e "Ubuntu 16.04 x86_64"
                        --project-version 2.3 
                        --project-milestone "2.3 M2" 
                        --project-date 2017-03-09 
                        --project-revision "rc1-7e37e6178358c9868bf0a62419f0e4d6c693d531" 
                        --verbose

Update a Test Run

./testopia_update.py 
                        --config testopia_update.config
                        -a update
                        -p ADT 
                        -c AUTO 
                        -b master 
                        -e "Ubuntu 16.04 x86_64"
                        --project-version 2.3 
                        --project-milestone "2.3 M2" 
                        --project-date 2017-03-09 
                        --project-revision "rc1-7e37e6178358c9868bf0a62419f0e4d6c693d531" 
                        --results-log results.log
                        --verbose

Proposed tools

This section is intended to add new tools to be implemented

How To Contribute

Contributing to any of the yoctoproject.org repositories, including qa-tools, can be achieved with the Poky Contributions guidelines.

Mantainer Anibal Limon