PR Service - Revision history

Stephano: /* Exmaple exported metadata */

2019-08-20T03:49:44Z

Exmaple exported metadata

← Older revision		Revision as of 03:49, 20 August 2019
Line 114:		Line 114:
	* ParseCompleted Event: export all meta-information (the structure of the database table used) to PRSERV_DUMPFILE by calling: oe.prservice.prserv_export_tofile(d, metainfo, None, True) is called		* ParseCompleted Event: export all meta-information (the structure of the database table used) to PRSERV_DUMPFILE by calling: oe.prservice.prserv_export_tofile(d, metainfo, None, True) is called

	=== ~~Exmaple~~ exported metadata ===		=== Example exported metadata ===

	#Table: PRMAIN_nohist		#Table: PRMAIN_nohist

Rpurdie: /* bitbake-prserv-tool */

2013-01-31T13:07:54Z

bitbake-prserv-tool

← Older revision		Revision as of 13:07, 31 January 2013
Line 64:		Line 64:

	bitbake-prserv-tool migrate_localcount		bitbake-prserv-tool migrate_localcount

			(this needs to be executed once per MACHINE)

	== Technical Details ==		== Technical Details ==

Constantin Musca: /* bitbake-prserv-tool */

2013-01-30T09:02:02Z

bitbake-prserv-tool

← Older revision		Revision as of 09:02, 30 January 2013
Line 60:		Line 60:
	bitbake-prserv-tool import <file>		bitbake-prserv-tool import <file>
	bitbake-prserv-tool export <file>		bitbake-prserv-tool export <file>

			In addition, bitbake-prserv-tool can be used to migrate git LOCALCOUNTs to AUTOINCs:

	bitbake-prserv-tool migrate_localcount		bitbake-prserv-tool migrate_localcount

Constantin Musca: /* prserv.bbclass */

2013-01-30T09:00:43Z

prserv.bbclass

← Older revision		Revision as of 09:00, 30 January 2013
Line 90:		Line 90:
	=== prserv.bbclass ===		=== prserv.bbclass ===

	* defines the prserv_get_pr_auto ~~(guarded by USE_PR_SERV)~~ which fetches the PR from the PR Server		* defines the prserv_get_pr_auto which fetches the PR from the PR Server
	* version <- PRAUTOINX; pkgarch <- PACKAGE_ARCH; checksum <- BB_TASKHASH;		* version <- PRAUTOINX; pkgarch <- PACKAGE_ARCH; checksum <- BB_TASKHASH;
	* uses the __PRSERV_CONN to store the connection		* uses the __PRSERV_CONN to store the connection

Constantin Musca: /* bitbake-prserv */

2013-01-30T08:58:44Z

bitbake-prserv

← Older revision		Revision as of 08:58, 30 January 2013
Line 82:		Line 82:
	=== bitbake-prserv ===		=== bitbake-prserv ===

	* needs the following ~~variables~~ set: PRSERV_HOST~~, PRSERV_PORT~~		* needs the following variable set: PRSERV_HOST
	* USE_PR_SERV=1 guards (prserv_dump_db, prserv_import_db)
	* the PR Service can export the PR info to a file (PRSERV_DUMPFILE)		* the PR Service can export the PR info to a file (PRSERV_DUMPFILE)
	* PRSERV_LOCKDOWN can be used for releases (the PRs don't change anymore)		* PRSERV_LOCKDOWN can be used for releases (the PRs don't change anymore)

Constantin Musca: /* bitbake-prserv-tool */

2013-01-30T08:57:32Z

bitbake-prserv-tool

← Older revision		Revision as of 08:57, 30 January 2013
Line 60:		Line 60:
	bitbake-prserv-tool import <file>		bitbake-prserv-tool import <file>
	bitbake-prserv-tool export <file>		bitbake-prserv-tool export <file>
			bitbake-prserv-tool migrate_localcount

	== Technical Details ==		== Technical Details ==

Constantin Musca: /* Basic Operation */

2013-01-30T08:56:49Z

Basic Operation

← Older revision		Revision as of 08:56, 30 January 2013
Line 17:		Line 17:
	In its simplest form with only a single build generating a package feed, the PR service can be enabled with the following in local.conf:		In its simplest form with only a single build generating a package feed, the PR service can be enabled with the following in local.conf:

	PRSERV_HOST = "localhost"		PRSERV_HOST = "localhost:0"
	~~PRSERV_PORT = "~~0"
	~~USE_PR_SERV = "1~~"

	which will then also mean that packages have increasing PR values. Bitbake itself will take care of starting and stopping the server.		which will then also mean that packages have increasing PR values. Bitbake itself will take care of starting and stopping the server.

Rpurdie: /* Overview */

2012-12-05T15:14:36Z

Overview

← Older revision		Revision as of 15:14, 5 December 2012
Line 7:		Line 7:
	The values these fields take is highly dependent on the policies and procedures of a given distribution and package feed. For a long time we've tried to maintain these values in the metadata but this is error prone, inaccurate and causes problems for people submitting recipes. The alternative is to autogenerate increasing numbers, particularly the revision field and this is what the PR service aims to do.		The values these fields take is highly dependent on the policies and procedures of a given distribution and package feed. For a long time we've tried to maintain these values in the metadata but this is error prone, inaccurate and causes problems for people submitting recipes. The alternative is to autogenerate increasing numbers, particularly the revision field and this is what the PR service aims to do.

	It should be noted that the build system itself knows when things need to rebuild by using "signatures" which represent any given build. All the inputs into a given task are represented by this signature and if it changes, the output can potentially change so it is rebuilt. The build system itself therefore doesn't rely on the PR numbers. We can use these signatures to generate PR values though.		It should be noted that the build system itself knows when things need to rebuild by using "signatures" which represent any given build. All the inputs into a given task are represented by this signature and if it changes, the output can potentially change so it is rebuilt. The build system itself therefore doesn't rely on the PR numbers. We can use these signatures to generate PR values though. The service works with both OEBasic and OEBasicHash generators, with the understanding that PR bumps happen when the checksum changes and the different generator mechanisms change signatures under different circumstances.

			As implemented, values from the PR service are included into the PR field as an addition of the form ".X" so r0 becomes r0.1, r0.2 and so on. This allows existing PR values to be used for whatever reasons allowing manual PR bumps should it be necessary.

	== Basic Operation ==		== Basic Operation ==

Rpurdie at 14:08, 5 December 2012

2012-12-05T14:08:07Z

← Older revision		Revision as of 14:08, 5 December 2012
Line 3:		Line 3:
	== Overview ==		== Overview ==

	One of the challenges the build system and its users face is how to maintain a package feed which is compatible with existing package manager ~~applciations~~ like rpm, apt and opkg. The main requirement is that versions increase in a linear fashion and there are a number of version components that ~~facilitiate~~ this, namely in order of decreasing priority PE, PV and PR (epoch, version and revision).		One of the challenges the build system and its users face is how to maintain a package feed which is compatible with existing package manager applications like rpm, apt and opkg. The main requirement is that versions increase in a linear fashion and there are a number of version components that facilitate this, namely in order of decreasing priority PE, PV and PR (epoch, version and revision).

	The values these fields take is highly dependent on the policies and procedures of a given distribution and package feed. For a long time we've tried to maintain these values in the metadata but this is error prone, inaccurate and causes problems for people submitting recipes. The alternative is to autogenerate increasing numbers, particularly the revision field and this is what the PR service aims to do.		The values these fields take is highly dependent on the policies and procedures of a given distribution and package feed. For a long time we've tried to maintain these values in the metadata but this is error prone, inaccurate and causes problems for people submitting recipes. The alternative is to autogenerate increasing numbers, particularly the revision field and this is what the PR service aims to do.

	It should be noted that the build system itself knows when things need to rebuild by using "~~singatures~~" which represent any given build. All the inputs into a given task are represented by this signature and if it changes, the output can potentially change so it is rebuilt. The build system itself therefore doesn't rely on the PR numbers. We can use these signatures to generate PR values though.		It should be noted that the build system itself knows when things need to rebuild by using "signatures" which represent any given build. All the inputs into a given task are represented by this signature and if it changes, the output can potentially change so it is rebuilt. The build system itself therefore doesn't rely on the PR numbers. We can use these signatures to generate PR values though.

	== Basic Operation ==		== Basic Operation ==

	By default, there is no PR server and the packages generated are just "self ~~concistent~~". The build system will add/remove any packages and there are no guarantees about upgrade paths.		By default, there is no PR server and the packages generated are just "self consistent". The build system will add/remove any packages and there are no guarantees about upgrade paths.

	In its simplest form with only a single build generating a package feed, the PR service can be enabled with the following in local.conf:		In its simplest form with only a single build generating a package feed, the PR service can be enabled with the following in local.conf:
Line 23:		Line 23:
	For more complex setups where multiple builds are working against a common package feed, you'd have one PR service running and connect each builder to it. This is done with a command like "bitbake-prserv --host <ip> --port <port> --start" and then the configuration would point at the server address and port number as needed.		For more complex setups where multiple builds are working against a common package feed, you'd have one PR service running and connect each builder to it. This is done with a command like "bitbake-prserv --host <ip> --port <port> --start" and then the configuration would point at the server address and port number as needed.

	Its recommended the server be used in ~~conjuction~~ with buildhistory which adds some sanity checks to package versions. This is enabled with:		Its recommended the server be used in conjunction with buildhistory which adds some sanity checks to package versions. This is enabled with:

	# It is recommended to activate "buildhistory" for testing the PR service		# It is recommended to activate "buildhistory" for testing the PR service
	INHERIT += "buildhistory"		INHERIT += "buildhistory"
	BUILDHISTORY_COMMIT = "1"		BUILDHISTORY_COMMIT = "1"

			== Interaction with sstate ==

			It should be noted that the PR information is not maintained as part of the sstate packages. If you maintain a sstate feed, its expected that either you run builders contributing to the sstate feed with a shared PR service, or you don't run one. Mixing the two will lead to problems.

	== Commands ==		== Commands ==
Line 59:		Line 63:
	== Technical Details ==		== Technical Details ==

	The basic scenario of using the network based PR service is that the client asks the server for a value (i.e. PRAUTO) by including the task checksum as part of the search index. The server returns the stored value if the search index is found in its backend database. If the search index is ~~notfound~~, the server bumps the value, stores the new index into the database, and returns the incremented value to the client. The client then includes the PRAUTO as part of the 'package revision' field value in the created package feed. The service is network based so a central database can be used to provide multiple build clients with the same value for the same index. The revision is injected into the metadata using the PKGR variable at package generation time.		The basic scenario of using the network based PR service is that the client asks the server for a value (i.e. PRAUTO) by including the task checksum as part of the search index. The server returns the stored value if the search index is found in its backend database. If the search index is not found, the server bumps the value, stores the new index into the database, and returns the incremented value to the client. The client then includes the PRAUTO as part of the 'package revision' field value in the created package feed. The service is network based so a central database can be used to provide multiple build clients with the same value for the same index. The revision is injected into the metadata using the PKGR variable at package generation time.

	* bitbake-prserv utility locks using the /tmp/PRServer_hostaddr_hostport.pid file		* bitbake-prserv utility locks using the /tmp/PRServer_hostaddr_hostport.pid file

Rpurdie: /* = bitbake-prserv */

2012-12-05T13:57:41Z

= bitbake-prserv

← Older revision		Revision as of 13:57, 5 December 2012
Line 75:		Line 75:
	== Other Observations ==		== Other Observations ==

	=== bitbake-prserv ==		=== bitbake-prserv ===

	* needs the following variables set: PRSERV_HOST, PRSERV_PORT		* needs the following variables set: PRSERV_HOST, PRSERV_PORT