MediaWiki Reporting Extension

Summary

This page describes the functionality and usage instructions for the MediaWiki Testopia reporting extension. Derived from http://www.mediawiki.org/wiki/Extension:Testopia_Reports this extension was modified in order to generate reports specific to the Yocto Project's needs. If you wish to use it on another MediaWiki server you can do so by cloning it from the public repository: https://github.com/halstead/testopiareports .

Report types

This is a list of reports that can be generated using the reporting extension. The parameters that you need in order to generate a report are:

1. report_id = the ID of the report type (ex: testplan, bugs, status, agenda, concatstats) - does not support multiple inputs
2. plan_id = the ID of the test plan/test plans - supports multiple inputs separated by comma + space ( ", " )
3. run_id = the ID of the test run/test runs - supports multiple inputs separated by comma + space ( ", " )

Testplan

Generates a list of test cases from one or more test plans.

NOTE: this is currently the only report type that supports (only) test plans. All the rest support (only) test runs.

Here is an example:

Code	Output
{{#testopia: |report_id=testplan |plan_id=34}}	{{#testopia:	report_id=testplan	plan_id=34}}

Status

Generates a status report for test cases in one or more test runs. It can also create a chart from the result as seen below:

Code	Output
{{#testopia: |report_id=status |run_id=411 |charttype=pie3}}	{{#testopia:	report_id=status	run_id=411	charttype=pie3}}

Bugs

Generates bug reports for one or more test runs. It has the following special fields:

1. Age(days) - the number of days the bug is opened, relevant to the test run's date field (this is extracted from the test run's name using this format: YYYY-MM-DD. This date must be placed at the begining of the Test Run's name)
2. Test Runs - links to all test runs that the bug was attached (filters are applicable)

Code	Output
{{#testopia: |report_id=bugs |run_id=403, 415, 416, 417, 404 |chart=bstatus,prio |charttype=pie,pie}}	{{#testopia:	report_id=bugs	run_id=403, 415, 416, 417, 404	chart=bstatus,prio	charttype=pie,pie}}

Agenda

Generates a detailed report of test cases in one or more test runs. If more that 1 test runs are introduced 3 new columns will be added: Test Run, Test Plan and Environment (this is done in order to differentiate between test runs)

Code	Output
{{#testopia: |report_id=agenda |run_id=411}}	{{#testopia:	report_id=agenda	run_id=411}}

ConcatStats

Generates a one-test_run-per-line report. There are a few things to note about how this report works:

1. test runs with identical Test Plans and Environments are merged together and the % is calculated based on the total number of test cases. The test run links are aggregated in the Test Run field.
2. when calculating the % only PASSED/BLOCKED/FAILED test cases are taken into account. This means that all other states are ignored(IDLE, PAUSED, RUNNING, ERROR, etc.). This is done because at the moment we use this report only for finished test suites. This may be changed in the future.

Code	Output
{{#testopia: |report_id=concatstats |run_id=403, 415, 416, 417, 404, 411}}	{{#testopia:	report_id=concatstats	run_id=403, 415, 416, 417, 404, 411}}

Other parameters

The reports use other parameters in order to filter or add functionality.

Functionality parameters

Note: The following table was adapted from the extension's original site

Parameter	Possible Values	Description
charttype	Google Chart API: pie - a 2D pie chart pie3 - a 3D bar chart bar - a 2D bar chart meter - the Google Meter	Chart type to render. This parameter might no be supported by all reports.
chart	depending on the report status: no specification needed bugs: bstatus, prio testplan: category agenda: -	If a report is capable of rendering more than one dimension of the table, this could be specified by this parameter. If more than chart should be rendered, the charts must be comma separated. Example: status,prio
chartlayout	horizontal,vertical, <number>	Horizontal or vertical alignment of charts. If <number> is used this is specifying the number of colums for charts. E.g. 2 will create a grid with two columns and x rows of charts.
chartpos	top,right,bottom,left	The position of the charts relative to the data table. Default: right
nudechart	true or false	Setting to true will display the chart(s) only and completely nude (without any surrounding tables)
showhide	true or false	A button will be displayed to hide or show the report. This requires JavaScript to be enabled.
hidden	true or false	If using parameter "showhide" this defines the initial state of the report. If set to true the report will be hidden when the pages is opened.
chartwidth	size in pixel or auto	If more than chart is displayed the widths could be separated by a comma. If less widths are specified than charts the last width will be used for the remainders. Using "auto" will use the build-in default or it will try to calculate it.
chartheight	size in pixel or auto	If more than chart is displayed the heights could be separated by a comma. If less heights are specified than charts the last height will be used for the remainders. Using "auto" will use the build-in default or it will try to calculate it.
total	true or false	Showing a line with a total below the report
sortable	true or false	Using the MediaWiki table class "sortable"
zebra	true or false	If true alternative line color will be used
hidetable	true or false	When rendering a chart, this parameter is hiding the data table.
debug	true or false	This will display some additional data for finding bugs in this extension.
title	true or false	If true the charts will have a title. Default is true.
roundperc	numeric value	Number of decimals to that a percentage will be rounded. The default is 0.

Filtering parameters

By adding "run_id=-1" and using the following parameters the reports can be automated:

Parameter	What it does	Report types that use it
product	Accepts strings separated by comma + space ( ", " ) that filter by the Product of the test run. If the test run's summary has ANY of the values it will be passed further.	agenda, bugs, status, concatstats
testplan	Accepts strings separated by comma + space ( ", " ) that filter by the Test Plan of the test run. If the test run's summary has ANY of the values it will be passed further.	agenda, bugs, status, concatstats
environment	Accepts strings separated by comma + space ( ", " ) that filter by the Environment of the test run. If the test run's summary has ANY of the values it will be passed further.	agenda, bugs, status, concatstats
build	Accepts strings separated by comma + space ( ", " ) that filter by the Build of the test run. If the test run's summary has ANY of the values it will be passed further.	agenda, bugs, status, concatstats
environment	Accepts strings separated by comma + space ( ", " ) that filter by the Environment of the test run. If the test run's summary has ANY of the values it will be passed further.	agenda, bugs, status, concatstats
insummary_any	Accepts strings separated by comma + space ( ", " ) that filter by the Summary of the test run. If the test run's summary has ANY of the values it will be passed further.	agenda, bugs, status, concatstats
insummary_all	Accepts strings separated by comma + space ( ", " ) that filter by the Summary of the test run. If the test run's summary has ALL of the values it will be passed further.	agenda, bugs, status, concatstats

Note: insummary_all and insummary_any accept a special value called "max". This is replaced by the date of the latest test suite after the "testplan" filter is applied. This is useful to completely automate report generation for the latest test suite performed for a specific test plan. An example can be found here: https://wiki.yoctoproject.org/wiki/index.php?title=1.3_QA_Status