MediaWiki Reporting Extension: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
=Summary=
=Summary=


This page describes the functionality and usage instructions for the Testopia MediaWiki reporting extension. Derived from http://www.mediawiki.org/wiki/Extension:Testopia_Reports this extension was modified in order to generate reports specific to the Yocto Project's needs. If you wish to use it on another MediaWiki server you can do so by cloning it from the public repository: https://github.com/halstead/testopiareports .
This page describes the functionality and usage instructions for the MediaWiki Testopia reporting extension. Derived from http://www.mediawiki.org/wiki/Extension:Testopia_Reports this extension was modified in order to generate reports specific to the Yocto Project's needs. If you wish to use it on another MediaWiki server you can do so by cloning it from the public repository: https://github.com/halstead/testopiareports .


=Report types=
=Report types=


This is a list of reports that can be generated using the reporting extension. The parameters that you need in order to generate a raport are:
This is a list of reports that can be generated using the reporting extension. The parameters that you need in order to generate a report are:
# report_id = the ID of the report type (ex: testplan, bugs, status, agenda, concatstats) - does not support multiple inputs
# report_id = the ID of the report type (ex: testplan, bugs, status, agenda, concatstats) - does not support multiple inputs
# plan_id = the ID of the test plan/test plans  - supports multiple inputs separated by comma + space ( ", " )
# plan_id = the ID of the test plan/test plans  - supports multiple inputs separated by comma + space ( ", " )
Line 14: Line 14:
Generates a list of test cases from one or more test plans.  
Generates a list of test cases from one or more test plans.  


NOTE: this is the only current report type that supports (only) test plans. All the rest support (only) test runs.
NOTE: this is currently the only report type that supports (only) test plans. All the rest support (only) test runs.


Here is an example:
Here is an example:
Line 80: Line 80:


==Agenda==
==Agenda==
Generates a detailed report of test cases in one or more test runs. If more that 1 test runs are introduced 3 new columns will be added: Test Run, Test Plan and Environment (this is done in order to differentiate between test runs)


{| class="wikitable"
{| class="wikitable"
Line 95: Line 97:
|run_id=411}}
|run_id=411}}
|}
|}
==ConcatStats==
Generates a one-test_run-per-line report. There are a few things to note about how this report works:
# test runs with identical Test Plans and Environments are merged together and the % is calculated based on the total number of test cases. The test run links are aggregated in the Test Run field.
# when calculating the % only PASSED/BLOCKED/FAILED test cases are taken into account. This means that all other states are ignored(IDLE, PAUSED, RUNNING, ERROR, etc.). This is done because at the moment we use this report only for finished test suites. This may be changed in the future.
{| class="wikitable"
|-
|Code
|Output
|-
| <pre>
{{#testopia:
|report_id=concatstats
|run_id=403, 415, 416, 417, 404, 411}}
</pre>
|{{#testopia:
|report_id=concatstats
|run_id=403, 415, 416, 417, 404, 411}}
|}
=Other parameters=
The reports use other parameters in order to filter or add functionality.
==Functionality parameters==
Note: The following table was adapted from the extension's original site
{| class="wikitable sortable"
|-
!Parameter
! class="unsortable" | Possible Values
! class="unsortable" | Description
|-
|charttype
|
*Google Chart API:
**pie - a 2D pie chart
**pie3 - a 3D bar chart
**bar - a 2D bar chart
**meter - the Google Meter
|Chart type to render. This parameter might no be supported by all reports.
|-
|chart
|depending on the report
* '''status''': no specification needed
* '''bugs''': bstatus, prio
* '''testplan''': category
* '''agenda''': -
|If a report is capable of rendering more than one dimension of the table, this could be specified by this parameter. If more than chart should be rendered, the charts must be comma separated. Example: status,prio
|-
|chartlayout
|horizontal,vertical, <number>
|Horizontal or vertical alignment of charts. If <number> is used this is specifying the number of colums for charts. E.g. 2 will create a grid with two columns and x rows of charts.
|-
|chartpos
|top,right,bottom,left
|The position of the charts relative to the data table. Default: right
|-
|nudechart
|true or false
|Setting to true will display the chart(s) only and completely nude (without any surrounding tables)
|-
|showhide
|true or false
|A button will be displayed to hide or show the report. This requires JavaScript to be enabled.
|-
|hidden
|true or false
|If using parameter "showhide" this defines the initial state of the report. If set to true the report will be hidden when the pages is opened.
|-
|chartwidth
|size in pixel or auto
|If more than chart is displayed the widths could be separated by a comma. If less widths are specified than charts the last width will be used for the remainders. Using "auto" will use the build-in default or it will try to calculate it.
|-
|chartheight
|size in pixel or auto
|If more than chart is displayed the heights could be separated by a comma. If less heights are specified than charts the last height will be used for the remainders. Using "auto" will use the build-in default or it will try to calculate it.
|-
|total
|true or false
|Showing a line with a total below the report
|-
|sortable
|true or false
|Using the MediaWiki table class "sortable"
|-
|zebra
|true or false
|If true alternative line color will be used
|-
|hidetable
|true or false
|When rendering a chart, this parameter is hiding the data table.
|-
|debug
|true or false
|This will display some additional data for finding bugs in this extension.
|-
| title
| true or false
| If true the charts will have a title. Default is true.
|-
| roundperc
| numeric value
| Number of decimals to that a percentage will be rounded. The default is 0.
|}
==Filtering parameters==
By adding "run_id=-1" and using the following parameters the reports can be automated:
{| class="wikitable sortable"
|-
!Parameter
! class="unsortable" | What it does
! class="unsortable" | Report types that use it
|-
|product
|Accepts strings separated by comma + space ( ", " ) that filter by the Product of the test run. If the test run's summary has ANY of the values it will be passed further.
|agenda, bugs, status, concatstats
|-
|testplan
|Accepts strings separated by comma + space ( ", " ) that filter by the Test Plan of the test run. If the test run's summary has ANY of the values it will be passed further.
|agenda, bugs, status, concatstats
|-
|environment
|Accepts strings separated by comma + space ( ", " ) that filter by the Environment of the test run. If the test run's summary has ANY of the values it will be passed further.
|agenda, bugs, status, concatstats
|-
|build
|Accepts strings separated by comma + space ( ", " ) that filter by the Build of the test run. If the test run's summary has ANY of the values it will be passed further.
|agenda, bugs, status, concatstats
|-
|environment
|Accepts strings separated by comma + space ( ", " ) that filter by the Environment of the test run. If the test run's summary has ANY of the values it will be passed further.
|agenda, bugs, status, concatstats
|-
|insummary_any
|Accepts strings separated by comma + space ( ", " ) that filter by the Summary of the test run. If the test run's summary has ANY of the values it will be passed further.
|agenda, bugs, status, concatstats
|-
|insummary_all
|Accepts strings separated by comma + space ( ", " ) that filter by the Summary of the test run. If the test run's summary has ALL of the values it will be passed further.
|agenda, bugs, status, concatstats
|-
|}
Note: insummary_all and insummary_any accept a special value called "max". This is replaced by the date of the latest test suite after the "testplan" filter is applied. This is useful to completely automate report generation for the latest test suite performed for a specific test plan. An example can be found here: https://wiki.yoctoproject.org/wiki/index.php?title=1.3_QA_Status

Latest revision as of 09:13, 13 May 2013

Summary

This page describes the functionality and usage instructions for the MediaWiki Testopia reporting extension. Derived from http://www.mediawiki.org/wiki/Extension:Testopia_Reports this extension was modified in order to generate reports specific to the Yocto Project's needs. If you wish to use it on another MediaWiki server you can do so by cloning it from the public repository: https://github.com/halstead/testopiareports .

Report types

This is a list of reports that can be generated using the reporting extension. The parameters that you need in order to generate a report are:

  1. report_id = the ID of the report type (ex: testplan, bugs, status, agenda, concatstats) - does not support multiple inputs
  2. plan_id = the ID of the test plan/test plans - supports multiple inputs separated by comma + space ( ", " )
  3. run_id = the ID of the test run/test runs - supports multiple inputs separated by comma + space ( ", " )

Testplan

Generates a list of test cases from one or more test plans.

NOTE: this is currently the only report type that supports (only) test plans. All the rest support (only) test runs.

Here is an example:

Code Output
 
{{#testopia:
|report_id=testplan
|plan_id=34}}
{{#testopia: report_id=testplan plan_id=34}}

Status

Generates a status report for test cases in one or more test runs. It can also create a chart from the result as seen below:

Code Output
 
{{#testopia:
|report_id=status
|run_id=411
|charttype=pie3}}
{{#testopia: report_id=status run_id=411 charttype=pie3}}

Bugs

Generates bug reports for one or more test runs. It has the following special fields:

  1. Age(days) - the number of days the bug is opened, relevant to the test run's date field (this is extracted from the test run's name using this format: YYYY-MM-DD. This date must be placed at the begining of the Test Run's name)
  2. Test Runs - links to all test runs that the bug was attached (filters are applicable)
Code Output
 
{{#testopia:
|report_id=bugs
|run_id=403, 415, 416, 417, 404
|chart=bstatus,prio
|charttype=pie,pie}}
{{#testopia: report_id=bugs run_id=403, 415, 416, 417, 404 chart=bstatus,prio charttype=pie,pie}}

Agenda

Generates a detailed report of test cases in one or more test runs. If more that 1 test runs are introduced 3 new columns will be added: Test Run, Test Plan and Environment (this is done in order to differentiate between test runs)

Code Output
 
{{#testopia:
|report_id=agenda
|run_id=411}}
{{#testopia: report_id=agenda run_id=411}}

ConcatStats

Generates a one-test_run-per-line report. There are a few things to note about how this report works:

  1. test runs with identical Test Plans and Environments are merged together and the % is calculated based on the total number of test cases. The test run links are aggregated in the Test Run field.
  2. when calculating the % only PASSED/BLOCKED/FAILED test cases are taken into account. This means that all other states are ignored(IDLE, PAUSED, RUNNING, ERROR, etc.). This is done because at the moment we use this report only for finished test suites. This may be changed in the future.
Code Output
 
{{#testopia:
|report_id=concatstats
|run_id=403, 415, 416, 417, 404, 411}}
{{#testopia: report_id=concatstats run_id=403, 415, 416, 417, 404, 411}}

Other parameters

The reports use other parameters in order to filter or add functionality.

Functionality parameters

Note: The following table was adapted from the extension's original site

Parameter Possible Values Description
charttype
  • Google Chart API:
    • pie - a 2D pie chart
    • pie3 - a 3D bar chart
    • bar - a 2D bar chart
    • meter - the Google Meter
Chart type to render. This parameter might no be supported by all reports.
chart depending on the report
  • status: no specification needed
  • bugs: bstatus, prio
  • testplan: category
  • agenda: -
If a report is capable of rendering more than one dimension of the table, this could be specified by this parameter. If more than chart should be rendered, the charts must be comma separated. Example: status,prio
chartlayout horizontal,vertical, <number> Horizontal or vertical alignment of charts. If <number> is used this is specifying the number of colums for charts. E.g. 2 will create a grid with two columns and x rows of charts.
chartpos top,right,bottom,left The position of the charts relative to the data table. Default: right
nudechart true or false Setting to true will display the chart(s) only and completely nude (without any surrounding tables)
showhide true or false A button will be displayed to hide or show the report. This requires JavaScript to be enabled.
hidden true or false If using parameter "showhide" this defines the initial state of the report. If set to true the report will be hidden when the pages is opened.
chartwidth size in pixel or auto If more than chart is displayed the widths could be separated by a comma. If less widths are specified than charts the last width will be used for the remainders. Using "auto" will use the build-in default or it will try to calculate it.
chartheight size in pixel or auto If more than chart is displayed the heights could be separated by a comma. If less heights are specified than charts the last height will be used for the remainders. Using "auto" will use the build-in default or it will try to calculate it.
total true or false Showing a line with a total below the report
sortable true or false Using the MediaWiki table class "sortable"
zebra true or false If true alternative line color will be used
hidetable true or false When rendering a chart, this parameter is hiding the data table.
debug true or false This will display some additional data for finding bugs in this extension.
title true or false If true the charts will have a title. Default is true.
roundperc numeric value Number of decimals to that a percentage will be rounded. The default is 0.

Filtering parameters

By adding "run_id=-1" and using the following parameters the reports can be automated:

Parameter What it does Report types that use it
product Accepts strings separated by comma + space ( ", " ) that filter by the Product of the test run. If the test run's summary has ANY of the values it will be passed further. agenda, bugs, status, concatstats
testplan Accepts strings separated by comma + space ( ", " ) that filter by the Test Plan of the test run. If the test run's summary has ANY of the values it will be passed further. agenda, bugs, status, concatstats
environment Accepts strings separated by comma + space ( ", " ) that filter by the Environment of the test run. If the test run's summary has ANY of the values it will be passed further. agenda, bugs, status, concatstats
build Accepts strings separated by comma + space ( ", " ) that filter by the Build of the test run. If the test run's summary has ANY of the values it will be passed further. agenda, bugs, status, concatstats
environment Accepts strings separated by comma + space ( ", " ) that filter by the Environment of the test run. If the test run's summary has ANY of the values it will be passed further. agenda, bugs, status, concatstats
insummary_any Accepts strings separated by comma + space ( ", " ) that filter by the Summary of the test run. If the test run's summary has ANY of the values it will be passed further. agenda, bugs, status, concatstats
insummary_all Accepts strings separated by comma + space ( ", " ) that filter by the Summary of the test run. If the test run's summary has ALL of the values it will be passed further. agenda, bugs, status, concatstats


Note: insummary_all and insummary_any accept a special value called "max". This is replaced by the date of the latest test suite after the "testplan" filter is applied. This is useful to completely automate report generation for the latest test suite performed for a specific test plan. An example can be found here: https://wiki.yoctoproject.org/wiki/index.php?title=1.3_QA_Status