Event Logger Data Interfaces - Revision history

Ddalex at 13:15, 15 October 2013

2013-10-15T13:15:27Z

← Older revision		Revision as of 13:15, 15 October 2013
Line 1:		Line 1:
	[[Category:~~WebHob~~]]		[[Category:Toaster]]

	Documenting the event logger data interfaces and action models.		Documenting the event logger data interfaces and action models.

Ddalex at 15:04, 10 June 2013

2013-06-10T15:04:40Z

← Older revision		Revision as of 15:04, 10 June 2013
Line 16:		Line 16:
	EVT will run resident in memory, connecting to the bitbake server and accepting events. It will have an internal state that will follow the internal state of the bitbake server. This allows the EVT to infer information about the server state, and export this information to the DSI.		EVT will run resident in memory, connecting to the bitbake server and accepting events. It will have an internal state that will follow the internal state of the bitbake server. This allows the EVT to infer information about the server state, and export this information to the DSI.

	The summary of data that is to be retrieve is here [Web_Hob_Event_information].		The summary of data that is to be retrieve is here [[Web_Hob_Event_information]].

	The event list that it receives from Bitbake is documented here, together with standard actions being taken in Knotty and what should happen in EVT.		The event list that it receives from Bitbake is documented here, together with standard actions being taken in Knotty and what should happen in EVT.

Ddalex at 15:04, 10 June 2013

2013-06-10T15:04:25Z

← Older revision		Revision as of 15:04, 10 June 2013
Line 15:		Line 15:

	EVT will run resident in memory, connecting to the bitbake server and accepting events. It will have an internal state that will follow the internal state of the bitbake server. This allows the EVT to infer information about the server state, and export this information to the DSI.		EVT will run resident in memory, connecting to the bitbake server and accepting events. It will have an internal state that will follow the internal state of the bitbake server. This allows the EVT to infer information about the server state, and export this information to the DSI.

			The summary of data that is to be retrieve is here [Web_Hob_Event_information].

	The event list that it receives from Bitbake is documented here, together with standard actions being taken in Knotty and what should happen in EVT.		The event list that it receives from Bitbake is documented here, together with standard actions being taken in Knotty and what should happen in EVT.

Calin Dragomir: /* interface with the Data Store system */

2013-06-07T14:33:24Z

interface with the Data Store system

← Older revision		Revision as of 14:33, 7 June 2013
Line 143:		Line 143:

	The data interface is based on the Django ORM.		The data interface is based on the Django ORM.
	A definition of [[Django models]] for build and task information is available.		A definition of [[Django models]] for build and task information is available.

			A list with the [[Event information]] we currently have and the things we need to have is also available.

	== Artifacts analysis module ==		== Artifacts analysis module ==

	Once a Bitbake build is completed, it's artifacts are available for usage, inspection and download. The artifacts analysis module (AAM) will be launched by EVT at the end of a build, parse the generated artifacts and insert relevant data into the DSI.		Once a Bitbake build is completed, it's artifacts are available for usage, inspection and download. The artifacts analysis module (AAM) will be launched by EVT at the end of a build, parse the generated artifacts and insert relevant data into the DSI.

Calin Dragomir at 12:16, 7 June 2013

2013-06-07T12:16:09Z

← Older revision		Revision as of 12:16, 7 June 2013
Line 70:		Line 70:
	\|-		\|-
	\|bb.command.CommandFailed \|\|		\|bb.command.CommandFailed \|\|
	* Mark this as an error and exit the main loop by putting ~~shudown~~ into state 2		* Mark this as an error and exit the main loop by putting shutdown into state 2
	\|\|		\|\|
	\|-		\|-

Calin Dragomir at 08:21, 5 June 2013

2013-06-05T08:21:22Z

← Older revision		Revision as of 08:21, 5 June 2013
Line 18:		Line 18:
	The event list that it receives from Bitbake is documented here, together with standard actions being taken in Knotty and what should happen in EVT.		The event list that it receives from Bitbake is documented here, together with standard actions being taken in Knotty and what should happen in EVT.

	{\|		{\| class="wikitable"
	\|-		\|-
	! Event object type !! action taken in Knotty !! action in EVT \|\|		! Event object type !! action taken in Knotty !! action in EVT \|\|
	\|-		\|-
	\|bb.runqueue.runQueueExitWait \|\|		\|bb.runqueue.runQueueExitWait \|\|
			* Assigns 1 for main.shutdown if not already set
			\|\|
	\|-		\|-
	\|bb.event.LogExecTTY \|\|		\|bb.event.LogExecTTY \|\|
			*If the stdin and stdout are terminals -> while there are still tries:
			::if the event finishes successfully, the loop exists, and bb continues with other events.
			::else it waits for sleep_delay amount of time and retries.
			*A warning message is written in the logger file.
			\|\|
	\|-		\|-
	\|logging.LogRecord \|\|		\|logging.LogRecord \|\|
			*If there is an error or warning, count them
			*Log every event
			\|\|
	\|-		\|-
	\|bb.build.TaskFailed \|\|		\|bb.build.TaskFailed \|\|
			* If a logfile is found: display error by reading the logfile
			\|\|
	\|-		\|-
	\|bb.build.TaskBase \|\|		\|bb.build.TaskBase \|\|
			* Log the event message in the logger file
			\|\|
	\|-		\|-
	\|bb.event.ParseStarted \|\|		\|bb.event.ParseStarted \|\|
			* Start a new parseprogress if event.total is not zero
	\|-		\|-
	\|bb.event.ParseProgress \|\|		\|bb.event.ParseProgress \|\|
			* Update the parse progress based on the current event
			\|\|
	\|-		\|-
	\|bb.event.ParseCompleted \|\|		\|bb.event.ParseCompleted \|\|
			* If there is a parseprogress object, finish it and print to the console the results of the parse
			\|\|
	\|-		\|-
	\|bb.event.CacheLoadStarted \|\|		\|bb.event.CacheLoadStarted \|\|
			* Create a cacheprogress object
			\|\|
	\|-		\|-
	\|bb.event.CacheLoadProgress \|\|		\|bb.event.CacheLoadProgress \|\|
			* Update the cacheprogress based on the current event
			\|\|
	\|-		\|-
	\|bb.event.CacheLoadCompleted \|\|		\|bb.event.CacheLoadCompleted \|\|
			* Finish the current cacheprogress and print the number of events grabbed from dependency cache
			\|\|
	\|-		\|-
	\|bb.command.CommandFailed \|\|		\|bb.command.CommandFailed \|\|
			* Mark this as an error and exit the main loop by putting shudown into state 2
			\|\|
	\|-		\|-
	\|bb.command.CommandExit \|\|		\|bb.command.CommandExit \|\|
			* If a returned value is not set, grab the event exit code and set it as a return value
			\|\|
	\|-		\|-
	\|bb.command.CommandCompleted		\|bb.command.CommandCompleted \|\|
			* Put shutdown into state 2
			\|\|
	\|-		\|-
	\|bb.cooker.CookerExit \|\|		\|bb.cooker.CookerExit \|\|
			* Put shutdown into state 2
			\|\|
	\|-		\|-
	\|bb.event.MultipleProviders \|\|		\|bb.event.MultipleProviders \|\|
			* Log info about multiple providers available and instruct the user to define a PREFERRED_PROVIDER
			\|\|
	\|-		\|-
	\|bb.event.NoProvider \|\|		\|bb.event.NoProvider \|\|
			* Mark this as an error
			* Check dependees and log the error
			* List the error reasons in the log file
			\|\|
	\|-		\|-
	\|bb.runqueue.sceneQueueTaskStarted \|\|		\|bb.runqueue.sceneQueueTaskStarted \|\|
			* Marks the start of a scene queue task and logs info about it
			\|\|
	\|-		\|-
	\|bb.runqueue.runQueueTaskStarted \|\|		\|bb.runqueue.runQueueTaskStarted \|\|
			* Marks the start of a queue task and logs info about it
			\|\|
	\|-		\|-
	\|bb.runqueue.runQueueTaskFailed \|\|		\|bb.runqueue.runQueueTaskFailed \|\|
			* Grab failed task, append it to task failures and log the error together with the exit code
			\|\|
	\|-		\|-
	\|bb.runqueue.sceneQueueTaskFailed \|\|		\|bb.runqueue.sceneQueueTaskFailed \|\|
			* Log a warning about the setscene task that failed together with the exit code and report that the real task will be ran instead
			\|\|
	\|-		\|-
	\| bb.event.BuildBase \|\| ignored \|\| ignored		\| bb.event.BuildBase \|\| ignored \|\| ignored

Ddalex at 14:29, 31 May 2013

2013-05-31T14:29:20Z

← Older revision		Revision as of 14:29, 31 May 2013
Line 2:		Line 2:

	Documenting the event logger data interfaces and action models.		Documenting the event logger data interfaces and action models.
	The EVT is the Event Translation Layer, which receive a set of events from a Bitbake build run, and builds a in-memory record set describing the build.		* The EVT is the Event Translation Layer, which receive a set of events from a Bitbake build run, and builds a in-memory record set describing the build.
	The DSI is the Data Store Interface that will poll new records from each of the EVTs assigned to it, and write them in permanent record.		* The DSI is the Data Store Interface that will poll new records from each of the EVTs assigned to it, and write them in permanent record.
			* The AAM is the Artifacts Analysis Module. This module is run on build complete, scans the build/ directory for artifacts, and generates records sets about the build artifacts that will be stored in the data store.

	== EVT ==		== EVT ==

Ddalex at 14:24, 31 May 2013

2013-05-31T14:24:36Z

← Older revision		Revision as of 14:24, 31 May 2013
Line 2:		Line 2:

	Documenting the event logger data interfaces and action models.		Documenting the event logger data interfaces and action models.
	The EVT is the ~~EVent Tra~~		The EVT is the Event Translation Layer, which receive a set of events from a Bitbake build run, and builds a in-memory record set describing the build.
			The DSI is the Data Store Interface that will poll new records from each of the EVTs assigned to it, and write them in permanent record.

	== EVT ==		== EVT ==

	Bitbake uses a push-event model where it sends pickled Python objects over XMLRPC to an event sink.		Bitbake uses a push-event model where it sends pickled Python objects over XMLRPC to an event sink. EVT receives Python objects describing the progress of a build run, and updates its internal state to match the state of the Bitbake server. It will generate a set of records describing the statistics and results of the build run, e.g. task records for all the tasks that did run, build records for all targets, etc.

	=== interface with bitbake ===		=== interface with bitbake ===
Line 14:		Line 15:
	EVT will run resident in memory, connecting to the bitbake server and accepting events. It will have an internal state that will follow the internal state of the bitbake server. This allows the EVT to infer information about the server state, and export this information to the DSI.		EVT will run resident in memory, connecting to the bitbake server and accepting events. It will have an internal state that will follow the internal state of the bitbake server. This allows the EVT to infer information about the server state, and export this information to the DSI.

	The event list that it receives from Bitbake is documented here:		The event list that it receives from Bitbake is documented here, together with standard actions being taken in Knotty and what should happen in EVT.

	{\|		{\|
Line 90:		Line 91:

	=== interface with EVT ===		=== interface with EVT ===

			=== interface with the Data Store system ===

			The data interface is based on the Django ORM.
			A definition of [[Django models]] for build and task information is available.


			== Artifacts analysis module ==

			Once a Bitbake build is completed, it's artifacts are available for usage, inspection and download. The artifacts analysis module (AAM) will be launched by EVT at the end of a build, parse the generated artifacts and insert relevant data into the DSI.

Ddalex: Created page with "Category:WebHob Documenting the event logger data interfaces and action models. The EVT is the EVent Tra == EVT == Bitbake uses a push-event model where it sends pickled P..."

2013-05-31T14:16:43Z

Created page with "Category:WebHob Documenting the event logger data interfaces and action models. The EVT is the EVent Tra == EVT == Bitbake uses a push-event model where it sends pickled P..."

New page

[[Category:WebHob]]

Documenting the event logger data interfaces and action models.
The EVT is the EVent Tra

== EVT ==

Bitbake uses a push-event model where it sends pickled Python objects over XMLRPC to an event sink.

=== interface with bitbake ===

The bitbake server runs at IP:PORT location. A event server controller lives at IP:(PORT+2) accepting a XMLRPC interface with two commands: registerEventHandler and unregisterEventHandler.

EVT will run resident in memory, connecting to the bitbake server and accepting events. It will have an internal state that will follow the internal state of the bitbake server. This allows the EVT to infer information about the server state, and export this information to the DSI.

The event list that it receives from Bitbake is documented here:

{|
|-
! Event object type !! action taken in Knotty !! action in EVT ||
|-
|bb.runqueue.runQueueExitWait ||
|-
|bb.event.LogExecTTY ||
|-
|logging.LogRecord ||
|-
|bb.build.TaskFailed ||
|-
|bb.build.TaskBase ||
|-
|bb.event.ParseStarted ||
|-
|bb.event.ParseProgress ||
|-
|bb.event.ParseCompleted ||
|-
|bb.event.CacheLoadStarted ||
|-
|bb.event.CacheLoadProgress ||
|-
|bb.event.CacheLoadCompleted ||
|-
|bb.command.CommandFailed ||
|-
|bb.command.CommandExit ||
|-
|bb.command.CommandCompleted
|-
|bb.cooker.CookerExit ||
|-
|bb.event.MultipleProviders ||
|-
|bb.event.NoProvider ||
|-
|bb.runqueue.sceneQueueTaskStarted ||
|-
|bb.runqueue.runQueueTaskStarted ||
|-
|bb.runqueue.runQueueTaskFailed ||
|-
|bb.runqueue.sceneQueueTaskFailed ||
|-
| bb.event.BuildBase || ignored || ignored
|-
| bb.event.StampUpdate || ignored || ignored
|-
| bb.event.ConfigParsed || ignored || timestamp entry
|-
| bb.event.RecipeParsed || ignored || timestamp entry
|-
| bb.event.RecipePreFinalise || ignored || ignored
|-
| bb.runqueue.runQueueEvent || ignored
|-
| bb.runqueue.runQueueExitWait || ignored
|-
| bb.event.OperationStarted || ignored || timestamp
|-
| bb.event.OperationCompleted || ignored || timestamp, operation complete, timing
|-
| bb.event.OperationProgress || ignored || ignored
|-
| bb.event.DiskFull || ignored || error
|}

=== interface with DSI ===

== DSI ==

=== interface with EVT ===