Resulttool - Revision history

Yeoh, Ee Peng at 04:50, 7 April 2020

2020-04-07T04:50:22Z

← Older revision		Revision as of 04:50, 7 April 2020
Line 48:		Line 48:
	make the testcase configuration file to run a set of		make the testcase configuration file to run a set of
	test cases based on user selection		test cases based on user selection


	=== manualexecution example ===		=== manualexecution example ===
Line 74:		Line 73:
	-x EXECUTED_BY, --executed-by EXECUTED_BY		-x EXECUTED_BY, --executed-by EXECUTED_BY
	add executed-by configuration to each result file		add executed-by configuration to each result file


	=== merge example ===		=== merge example ===
Line 108:		Line 106:
	add extra test environment data to each result file		add extra test environment data to each result file
	configuration		configuration


	=== store example ===		=== store example ===
Line 118:		Line 115:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool store tmp/log/manual/ <yocto-testresults-dir>		$ resulttool store tmp/log/manual/ <yocto-testresults-dir>


	== regression ==		== regression ==
Line 183:		Line 181:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool regression-git <yocto-testresults-dir> --commit <commit-of-base> --commit2 <commit-of-target>		$ resulttool regression-git <yocto-testresults-dir> --commit <commit-of-base> --commit2 <commit-of-target>


	== report ==		== report ==
Line 212:		Line 211:
	are provided then output the selected test case result		are provided then output the selected test case result
	from the provided test result id		from the provided test result id


	=== report example ===		=== report example ===
Line 220:		Line 218:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool report <yocto-testresults-dir>		$ resulttool report <yocto-testresults-dir>


	== log ==		== log ==
Line 243:		Line 242:
	--raw-ptest show raw ptest log		--raw-ptest show raw ptest log
	--raw-reproducible show raw reproducible build logs		--raw-reproducible show raw reproducible build logs


	=== log example ===		=== log example ===

Yeoh, Ee Peng at 04:47, 7 April 2020

2020-04-07T04:47:34Z

@@ Line 31: / Line 31: @@
 arguments:
    file        specify path to manual test case JSON file.Note: Please use "" to encapsulate the file path.
 === manualexecution example ===
@@ Line 49: / Line 67: @@
    base_results    the results file/directory to import
    target_results  the target file or directory to merge the base_results with
 === merge example ===
@@ Line 77: / Line 103: @@
    -a, --all          include all files, not just testresults.json files
    -e, --allow-empty  don't error if no results to store are found
 === store example ===
@@ Line 163: / Line 195: @@
 options:
-   -h, --help         show this help message and exit
+   -h, --help            show this help message and exit
-   -t TAG, --tag TAG  source_dir is a git repository, report on the tag
+  --branch BRANCH, -B BRANCH
-                     specified from that repository
+                        Branch to find commit in
 === report example ===

Yeoh, Ee Peng at 04:43, 7 April 2020

2020-04-07T04:43:11Z

← Older revision		Revision as of 04:43, 7 April 2020
Line 173:		Line 173:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool report <yocto-testresults-dir>		$ resulttool report <yocto-testresults-dir>

			== log ==
			The log plugin was developed to print log available in a file/directory/URL.

			=== log --help ===
			show the logs from test results

			arguments:
			source_dir the results file/directory/URL to import

			options:
			-h, --help show this help message and exit
			--ptest PTEST show logs for a ptest
			--dump-ptest DIR Dump all ptest log files to the specified directory.
			--reproducible REPRODUCIBLE
			show logs for a reproducible test
			--prepend-run Dump ptest results to a subdirectory named after the
			test run when using --dump-ptest. Required if more
			than one test run is present in the result file
			--raw show raw (ptest) logs. Deprecated. Alias for "--raw-
			ptest"
			--raw-ptest show raw ptest log
			--raw-reproducible show raw reproducible build logs


			=== log example ===
			To dump ptest log to a directory.

			$ cd poky
			$ source oe-init-build-env
			$ resulttool log <yocto-testresults-dir> --dump-ptest <output-dir-for-ptest>

			To print custom ptest log from a custom ptest.

			$ cd poky
			$ source oe-init-build-env
			$ resulttool log <yocto-testresults-dir> --ptest <package-name-that-provide-ptest>

Yeoh, Ee Peng at 05:26, 6 April 2020

2020-04-06T05:26:50Z

← Older revision		Revision as of 05:26, 6 April 2020
Line 15:		Line 15:
	- "regression-git": regression git analysis		- "regression-git": regression git analysis
	- "report": summarise test results		- "report": summarise test results
			- "log": show logs

Yeoh, Ee Peng: /* regression-git */

2019-03-14T05:41:20Z

regression-git

← Older revision		Revision as of 05:41, 14 March 2019
Line 114:		Line 114:

	== regression-git ==		== regression-git ==
	The regression-git plugin was developed to provide regression analysis to test result available inside a local git repository.		The regression-git plugin was developed to provide regression analysis to test result available inside a local git repository. User need to clone yocto-testresults (https://git.yoctoproject.org/git/yocto-testresults) to local machine.

	=== regression-git --help ===		=== regression-git --help ===
Line 150:		Line 150:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool regression-git <yocto-testresults-dir> --commit <commit-of-base> --commit2 <commit-of-target>		$ resulttool regression-git <yocto-testresults-dir> --commit <commit-of-base> --commit2 <commit-of-target>


	== report ==		== report ==

Yeoh, Ee Peng: /* store example */

2019-03-14T05:40:06Z

store example

← Older revision		Revision as of 05:40, 14 March 2019
Line 80:		Line 80:
	To store tmp/log/manual/testresults.json to local result repository.		To store tmp/log/manual/testresults.json to local result repository.

	Assume "resulttool manualexecution" create the testresults.json file inside tmp/log/manual/testresults.json and yocto-testresults git repository was clone at <yocto-testresults-dir>.		Assume "resulttool manualexecution" create the testresults.json file inside tmp/log/manual/testresults.json and yocto-testresults local git repository was clone at <yocto-testresults-dir>.

	$ cd poky		$ cd poky

Yeoh, Ee Peng: /* store */

2019-03-14T05:39:33Z

store

← Older revision		Revision as of 05:39, 14 March 2019
Line 60:		Line 60:

	== store ==		== store ==
	The store plugin was developed to store testresults.json file and other artifacts files to local git repository. Where the local git repository serves as a database containing test result information to be used by "report", "regression" and "regression-git" plugin.		The store plugin was developed to store testresults.json file and other artifacts files to local git repository (https://git.yoctoproject.org/git/yocto-testresults). Where the local git repository serves as a database containing test result information to be used by "report", "regression" and "regression-git" plugin.

	=== store --help ===		=== store --help ===
Line 85:		Line 85:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool store tmp/log/manual/ <yocto-testresults-dir>		$ resulttool store tmp/log/manual/ <yocto-testresults-dir>


	== regression ==		== regression ==

Yeoh, Ee Peng at 07:01, 7 March 2019

2019-03-07T07:01:49Z

← Older revision		Revision as of 07:01, 7 March 2019
Line 4:		Line 4:

	To view the features available for resulttool, git clone poky repository, source poky environment, type "resulttool --help".		To view the features available for resulttool, git clone poky repository, source poky environment, type "resulttool --help".


	= List of features =		= List of features =
Line 14:		Line 15:
	- "regression-git": regression git analysis		- "regression-git": regression git analysis
	- "report": summarise test results		- "report": summarise test results


	== manualexecution ==		== manualexecution ==
Line 35:		Line 37:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool manualexecution ../meta/lib/oeqa/manual/sdk.json		$ resulttool manualexecution ../meta/lib/oeqa/manual/sdk.json


	== merge ==		== merge ==
Line 54:		Line 57:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool merge tmp/log/manual/testresults.json <target-testresults-dir>		$ resulttool merge tmp/log/manual/testresults.json <target-testresults-dir>


	== store ==		== store ==
Line 81:		Line 85:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool store tmp/log/manual/ <yocto-testresults-dir>		$ resulttool store tmp/log/manual/ <yocto-testresults-dir>


	== regression ==		== regression ==
Line 107:		Line 112:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool regression <base-result-dir> <target-result-dir>		$ resulttool regression <base-result-dir> <target-result-dir>


	== regression-git ==		== regression-git ==
Line 145:		Line 151:
	$ source oe-init-build-env		$ source oe-init-build-env
	$ resulttool regression-git <yocto-testresults-dir> --commit <commit-of-base> --commit2 <commit-of-target>		$ resulttool regression-git <yocto-testresults-dir> --commit <commit-of-base> --commit2 <commit-of-target>


	== report ==		== report ==

Yeoh, Ee Peng: /* merge */

2019-03-07T06:17:23Z

merge

← Older revision		Revision as of 06:17, 7 March 2019
Line 47:		Line 47:

	=== merge example ===		=== merge example ===
			To merge base testresults.json file to target directory.

			Assume base at tmp/log/manual/testresults.json and target directory at <target-testresults-dir>.

			$ cd poky
			$ source oe-init-build-env
			$ resulttool merge tmp/log/manual/testresults.json <target-testresults-dir>

	== store ==		== store ==

Yeoh, Ee Peng: /* manualexecution */

2019-03-07T06:02:01Z

manualexecution

← Older revision		Revision as of 06:02, 7 March 2019
Line 20:		Line 20:
	Current "Branch" and "Commit" used by the poky repository to run resulttool will be used to create the layers metadata inside testresults.json configuration. The layers metadata were important data required by the "resulttool store" for creating the git tag. Where the git tag will be used for "resulttool report" and "resulttool regression-git".		Current "Branch" and "Commit" used by the poky repository to run resulttool will be used to create the layers metadata inside testresults.json configuration. The layers metadata were important data required by the "resulttool store" for creating the git tag. Where the git tag will be used for "resulttool report" and "resulttool regression-git".

	The "MACHINE" and "IMAGE_BASENAME" configurations were mandatory inputs for manualexecution as these configurations will be used for "resulttool regression" and "resulttool regression-git", where regression used the configurations data to find matching result set for regression comparison if specific result ~~set~~ id was not provided.		The "MACHINE" and "IMAGE_BASENAME" configurations were mandatory inputs for manualexecution as these configurations will be used for "resulttool regression" and "resulttool regression-git", where regression used the configurations data to find matching result set for regression comparison if specific result id was not provided.

	=== manualexecution --help ===		=== manualexecution --help ===