QA/Test Report Tool

From Yocto Project
(Redirected from QA/Test Reporting Tool)
Jump to navigationJump to search

General Expectations

(add link share-ability of results to the list of requirements when it fits somewhere)

Here's a list of initial expectations of the Test Reporting Tool.

  1. The TRT must be able to receive test reports via a remote communication protocol.
    1. From expected sources of results (Auth)
    2. To existing test buckets (logical test belonging separation, could be by test component) through report sessions with:
      • ability to resume
      • ability to update existing results
      • ability to abort/discard
    3. Using existing test report protocols like XML
  2. The TRT should present an interface with views organized towards the following objectives:
    1. Test Buckets (browsing a component's test results history)
    2. Test Collections (The type of Milestone/Release or another defined collection)
  3. The TRT should be able to identify the type of result it is consuming.
    1. Boolean test results Passed/Failed
    2. Numeric test results (measurements)

QA Data Sources

There are several sources of quality data we'd like to be able to track in a QA dashboard:

  • oe-selftest results
  • bitbake-selftest results
  • build performance metrics
  • buildhistory trends
  • ???

Prior Art