QA/Test Report Tool: Difference between revisions

From Yocto Project
Jump to navigationJump to search
(Add some links to relevant prior art in visualising test results)
(Start to make a list of QA data sources)
Line 13: Line 13:
## Boolean test results Passed/Failed
## Boolean test results Passed/Failed
## Numeric test results (measurements)
## Numeric test results (measurements)
==QA Data Sources==
There are several sources of quality data we'd like to be able to track in a QA dashboard:
* oe-selftest results
* bitbake-selftest results
* build performance metrics
* buildhistory trends
* ???


==Prior Art==
==Prior Art==

Revision as of 20:40, 6 July 2016

General Expectations

Here's a list of initial expectations of the Test Reporting Tool.

  1. The TRT must be able to receive test reports via a remote communication protocol.
    1. From expected sources of results (Auth)
    2. To existing test buckets (logical test belonging separation, could be by test component)
    3. Using existing test report protocols like XML
  2. The TRT should present an interface with views organized towards the following objectives:
    1. Test Buckets (browsing a component's test results history)
    2. Test Collections (The type of Milestone/Release or another defined collection)
  3. The TRT should be able to identify the type of result it is consuming.
    1. Boolean test results Passed/Failed
    2. Numeric test results (measurements)

QA Data Sources

There are several sources of quality data we'd like to be able to track in a QA dashboard:

  • oe-selftest results
  • bitbake-selftest results
  • build performance metrics
  • buildhistory trends
  • ???

Prior Art