QA/Automated Test Report Template - Revision history

Markus Lehtonen: /* Execution Information Template */

2017-01-13T14:13:34Z

Execution Information Template

← Older revision		Revision as of 14:13, 13 January 2017
Line 26:		Line 26:
	<?xml version="1.0" ?>		<?xml version="1.0" ?>
	<metadata>		<metadata>
	<hostname>~~yctb01~~</hostname>		<hostname>mqz-osx-suse64</hostname>
	<~~host_distro name=Ubuntu"~~		<distro>
	~~version="16~~.~~04"~~>		<pretty_name>poky 2.2</pretty_name>
	~~Ubuntu 16~~.04.1 ~~LTS~~		<id>poky</id>
	</host_distro>		<version_id>2.2</version_id>
	<~~distro version~~="~~2.2~~"		</distro>
	~~git_branch="master"~~		<host_distro>
	~~git_commit="9e63f81c78e284c9b325fe04a1b59e61c7ad8a1a",~~		<id>opensuse</id>
	~~git_commit_count~~="~~43265~~">		<version_id>42.1</version_id>
	~~poky~~		<pretty_name>openSUSE Leap 42.1 (x86_64)</pretty_name>
	</~~distro~~>		</host_distro>
	<~~machine~~>~~qemux86~~-64</~~machine~~>		<layers>
	<~~!-- Only layers that are not part of distro or are custom revision -~~->		<layer name="meta">
	<layers>		<commit>c8bd57d7cf127b5ee315597408940870882a10e0</commit>
	<~~meta-selftest~~>		<commit_count>34619</commit_count>
	<branch>~~mariano~~/~~bug9954v2~~</~~branch~~>		<branch>feature/oeqa-metaxml</branch>
	<~~revision~~>~~43e652f3d1fee5ce7fad67e6400315eab1b34270~~</~~revision~~>		</layer>
	</~~meta~~-~~selftest~~>		<layer name="meta-selftest">
	</~~layers~~>		<commit>c8bd57d7cf127b5ee315597408940870882a10e0</commit>
			<commit_count>34619</commit_count>
			<branch>feature/oeqa-metaxml</branch>
			</layer>
			<layer name="meta-foobar"/>
			</layers>
			<bitbake>
			<commit>058f8517c041b80e8b591ad7d34a68281b2d03fc</commit>
			<commit_count>6577</commit_count>
			<branch>master</branch>
			</bitbake>
			<config>
			<variable name="BB_NUMBER_THREADS">2</variable>
			<variable name="MACHINE">qemux86</variable>
			<variable name="PARALLEL_MAKE">-j 2</variable>
			</config>
	</metadata>		</metadata>
	</pre>		</pre>

Markus Lehtonen: /* Execution Information Template */

2016-12-05T15:07:30Z

Execution Information Template

@@ Line 18: / Line 18: @@
 ** OS Version
 ** Hardware description
 And others that will be defined in the [[QA/Execution_Information_Template | Execution Information Template]]
 ==Report Template==

Markus Lehtonen: /* Performance Sample Report */

2016-12-05T14:06:02Z

Performance Sample Report

@@ Line 33: / Line 33: @@
 See a sample report of a [[QA/Sample_Report_oeqa.selftest.bbtests.BitbakeTests|QA test here]].
-===Performance Sample Report===
+===Build Performance Sample Report===
-WIP
+Proposed XML format for build performace reports. This follows the JUnit specification with the following extensions:

Benjamin at 23:45, 22 November 2016

2016-11-22T23:45:16Z

← Older revision		Revision as of 23:45, 22 November 2016
Line 8:		Line 8:
	Under the premise that most of the information in a performance and a testing report is alike, being the results notation the main difference; it should be feasible to use a single report specification that accounts for the aforementioned differences.		Under the premise that most of the information in a performance and a testing report is alike, being the results notation the main difference; it should be feasible to use a single report specification that accounts for the aforementioned differences.

	==JUnit-XML Report==		=JUnit-XML Performance/Testing Report=
	The QA testing has moved towards embracing the [http://llg.cubic.org/docs/junit/ JUnit-XML reporting standard] and there is work in progress to note the required extensions to use that format with performance measurements too.		The QA testing has moved towards embracing the [http://llg.cubic.org/docs/junit/ JUnit-XML reporting standard] and there is work in progress to note the required extensions to use that format with performance measurements too.

	==Test Report Template==		==Execution Information Template==

			There is a data set that will be common to all the test execution done to a subject, such as:

			* Test subject information
			** OS Version
			** Hardware description
			* Execution information
			** Date

			And others that will be defined in the [[QA/Execution_Information_Template \| Execution Information Template]]

			==Report Template==

	See the [[QA/JUnit_XML_Template \| test report template here]].		See the [[QA/JUnit_XML_Template \| test report template here]].
Line 17:		Line 29:
	This template is WIP to be extended for the performance measurements.		This template is WIP to be extended for the performance measurements.

	==QA Sample report==		===QA Sample report===

	See a sample report of a [[QA/Sample_Report_oeqa.selftest.bbtests.BitbakeTests\|QA test here]].		See a sample report of a [[QA/Sample_Report_oeqa.selftest.bbtests.BitbakeTests\|QA test here]].

	==Performance Sample Report==		===Performance Sample Report===

	WIP		WIP

Benjamin: Created page with "=Introduction= The test reporting in automation can enable further processes to be automated too. Specifically, results collection should be done by consumers that require no..."

2016-11-22T23:20:59Z

Created page with "=Introduction= The test reporting in automation can enable further processes to be automated too. Specifically, results collection should be done by consumers that require no..."

New page

=Introduction=

The test reporting in automation can enable further processes to be automated too. Specifically, results collection should be done by consumers that require no manual interaction to be fed with results data. Just as any other process flow, minimizing the number of input consumers is translated into immediate efficiency as the flowing data is standardized.

=Test and Performance Standard Report=
Performance measurements and software testing share steps in their processes but they have an important difference in the results they both provide: the performance results are expressed in scalar quantities while the testing output is expressed in a predefined set of results.

Under the premise that most of the information in a performance and a testing report is alike, being the results notation the main difference; it should be feasible to use a single report specification that accounts for the aforementioned differences.

==JUnit-XML Report==
The QA testing has moved towards embracing the [http://llg.cubic.org/docs/junit/ JUnit-XML reporting standard] and there is work in progress to note the required extensions to use that format with performance measurements too.

==Test Report Template==

See the [[QA/JUnit_XML_Template | test report template here]].

This template is WIP to be extended for the performance measurements.

==QA Sample report==

See a sample report of a [[QA/Sample_Report_oeqa.selftest.bbtests.BitbakeTests|QA test here]].

==Performance Sample Report==

WIP