QA/Automated Test Report Template: Difference between revisions

From Yocto Project
Jump to navigationJump to search
(Created page with "=Introduction= The test reporting in automation can enable further processes to be automated too. Specifically, results collection should be done by consumers that require no...")
 
No edit summary
Line 8: Line 8:
Under the premise that most of the information in a performance and a testing report is alike, being the results notation the main difference; it should be feasible to use a single report specification that accounts for the aforementioned differences.
Under the premise that most of the information in a performance and a testing report is alike, being the results notation the main difference; it should be feasible to use a single report specification that accounts for the aforementioned differences.


==JUnit-XML Report==
=JUnit-XML Performance/Testing Report=
The QA testing has moved towards embracing the [http://llg.cubic.org/docs/junit/ JUnit-XML reporting standard] and there is work in progress to note the required extensions to use that format with performance measurements too.
The QA testing has moved towards embracing the [http://llg.cubic.org/docs/junit/ JUnit-XML reporting standard] and there is work in progress to note the required extensions to use that format with performance measurements too.


==Test Report Template==
==Execution Information Template==
 
There is a data set that will be common to all the test execution done to a subject, such as:
 
* Test subject information
** OS Version
** Hardware description
* Execution information
** Date
 
And others that will be defined in the [[QA/Execution_Information_Template | Execution Information Template]]
 
==Report Template==


See the [[QA/JUnit_XML_Template | test report template here]].
See the [[QA/JUnit_XML_Template | test report template here]].
Line 17: Line 29:
This template is WIP to be extended for the performance measurements.
This template is WIP to be extended for the performance measurements.


==QA Sample report==
===QA Sample report===


See a sample report of a [[QA/Sample_Report_oeqa.selftest.bbtests.BitbakeTests|QA test here]].
See a sample report of a [[QA/Sample_Report_oeqa.selftest.bbtests.BitbakeTests|QA test here]].


==Performance Sample Report==
===Performance Sample Report===


WIP
WIP

Revision as of 23:45, 22 November 2016

Introduction

The test reporting in automation can enable further processes to be automated too. Specifically, results collection should be done by consumers that require no manual interaction to be fed with results data. Just as any other process flow, minimizing the number of input consumers is translated into immediate efficiency as the flowing data is standardized.

Test and Performance Standard Report

Performance measurements and software testing share steps in their processes but they have an important difference in the results they both provide: the performance results are expressed in scalar quantities while the testing output is expressed in a predefined set of results.

Under the premise that most of the information in a performance and a testing report is alike, being the results notation the main difference; it should be feasible to use a single report specification that accounts for the aforementioned differences.

JUnit-XML Performance/Testing Report

The QA testing has moved towards embracing the JUnit-XML reporting standard and there is work in progress to note the required extensions to use that format with performance measurements too.

Execution Information Template

There is a data set that will be common to all the test execution done to a subject, such as:

  • Test subject information
    • OS Version
    • Hardware description
  • Execution information
    • Date

And others that will be defined in the Execution Information Template

Report Template

See the test report template here.

This template is WIP to be extended for the performance measurements.

QA Sample report

See a sample report of a QA test here.

Performance Sample Report

WIP