Toaster 1.7 design - Revision history

Belenbarrospena at 15:47, 16 May 2014

2014-05-16T15:47:43Z

← Older revision		Revision as of 15:47, 16 May 2014
Line 1:		Line 1:
			[[Category:Toaster]]
	The design of the Toaster user interface takes place in iterations, and most recent designs supersede older ones.		The design of the Toaster user interface takes place in iterations, and most recent designs supersede older ones.

Belenbarrospena at 15:40, 16 May 2014

2014-05-16T15:40:19Z

← Older revision		Revision as of 15:40, 16 May 2014
Line 1:		Line 1:
			The design of the Toaster user interface takes place in iterations, and most recent designs supersede older ones.

			The latest iteration is always available at: http://www.yoctoproject.org/toaster

			Detailed design documentation is attached to [[Bugzilla feature list \| Bugzilla features]]. Features with design documentation attached have "GUI design available" written in the Whiteboard field, so it is easy to search for them.

	== Bugzilla feature list ==		== Bugzilla feature list ==

Belenbarrospena: /* Bugzilla feature list */

2014-05-16T14:33:56Z

Bugzilla feature list

← Older revision		Revision as of 14:33, 16 May 2014
Line 2:		Line 2:

	We have finished planning the Toaster scope for the Yocto Project 1.7 release. The outcome is [[Bugzilla feature list \| a set of Bugzilla features]]		We have finished planning the Toaster scope for the Yocto Project 1.7 release. The outcome is [[Bugzilla feature list \| a set of Bugzilla features]]

			The rest of this page will give you an idea of the scope definition process and discussions.

	== A couple of videos to kick off things ==		== A couple of videos to kick off things ==

Belenbarrospena at 14:28, 16 May 2014

2014-05-16T14:28:41Z

← Older revision		Revision as of 14:28, 16 May 2014
Line 1:		Line 1:
			== Bugzilla feature list ==

			We have finished planning the Toaster scope for the Yocto Project 1.7 release. The outcome is [[Bugzilla feature list \| a set of Bugzilla features]]

	== A couple of videos to kick off things ==		== A couple of videos to kick off things ==

Ddalex: /* Discussion */

2014-04-25T09:45:37Z

Discussion

← Older revision		Revision as of 09:45, 25 April 2014
Line 231:		Line 231:
	[BELEN] I am not sure what the answer is: we need input from Paul and Alex here. In my head, the project configuration (not the builds) is exported to some kind of file(s) that I can then share with others in whichever way I see fit (email, file sharing, etc).		[BELEN] I am not sure what the answer is: we need input from Paul and Alex here. In my head, the project configuration (not the builds) is exported to some kind of file(s) that I can then share with others in whichever way I see fit (email, file sharing, etc).

			[ALEX] I'm thinking that exporting the project will produce a file that can be imported on another Toaster instance and replicate the original settings of the project that was exported. The concrete means of achieving this may be as simple as exporting a JSON snapshot of the relevant database entries.

	=== Definition of a Project ===		=== Definition of a Project ===

Ddalex: /* Ability to identify build host and build directory */

2014-04-25T09:42:42Z

Ability to identify build host and build directory

← Older revision		Revision as of 09:42, 25 April 2014
Line 197:		Line 197:

	To support distributed build hosts sharing a common Toaster database, there needs to be a way in the database to identify which host (IP address and or host name) and local directory the project was built in.		To support distributed build hosts sharing a common Toaster database, there needs to be a way in the database to identify which host (IP address and or host name) and local directory the project was built in.


			===== Discussion =====

			[ALEX] I am not sure I follow the use case / user story here. Can we get more details on why this is important ?

	=== Ability to download files (YP #6096) ===		=== Ability to download files (YP #6096) ===

Ddalex: /* Feature list */

2014-04-25T09:30:53Z

Feature list

← Older revision		Revision as of 09:30, 25 April 2014
Line 11:		Line 11:

	This is the ability to use the Toaster interface to create new project directories, manage the content, and initiate the builds of selectable target images.		This is the ability to use the Toaster interface to create new project directories, manage the content, and initiate the builds of selectable target images.

			===== Summary of discussion =====

			* A project is a set of layers and configuration variables values that describes how to do a build.
			* Creating a project means writing up values for the above-mentioned set, and store these values in a persistent manner.
			* A build for a project means effecting (instantiating) the build process for the current set of values.
			* Build directory location (host/machine) is not relevant to Toaster.
			* Saving the build artifacts and providing access to the build artifacts is critical for Toaster.

	===== Discussion =====		===== Discussion =====

	[DAVE] What is a typical workflow that gets to the build project part? I see the configure screens but not the ~~actualy~~ build launch.		[DAVE] What is a typical workflow that gets to the build project part? I see the configure screens but not the actually build launch.

	[DAVE] Does clicking 'new build' fire off a build? If so, which configuration is built?		[DAVE] Does clicking 'new build' fire off a build? If so, which configuration is built?

Belenbarrospena: /* Discussion */

2014-04-22T14:34:29Z

Discussion

Belenbarrospena: /* Discussion */

2014-04-22T14:33:46Z

Discussion

← Older revision		Revision as of 14:33, 22 April 2014
Line 41:		Line 41:

	[BELEN] We will need to overcome my personal hate of 'reset to defaults' options :) They are very dangerous if we don't provide an 'undo' option as well: they might wipe your configuration and revert it to the rather useless default one, losing a bunch of work you did beforehand. Any configuration changes you make are saved: in that sense, they are 'sticky'. If I create project A, then I add 2 layers to the project configuration, select a machine and 3 targets, all those options are saved to the project configuration. You don't need to add the 2 layers, select the machine and the 3 targets every time you want to run a build.		[BELEN] We will need to overcome my personal hate of 'reset to defaults' options :) They are very dangerous if we don't provide an 'undo' option as well: they might wipe your configuration and revert it to the rather useless default one, losing a bunch of work you did beforehand. Any configuration changes you make are saved: in that sense, they are 'sticky'. If I create project A, then I add 2 layers to the project configuration, select a machine and 3 targets, all those options are saved to the project configuration. You don't need to add the 2 layers, select the machine and the 3 targets every time you want to run a build.

			[RAVI] I am wondering about the purpose of a project. In the design, project has a machine and owner attributes. If we define a project as a collection of builds for a machine, then we should not allow to change the machine after a project is created. Then, if a project is just a collection of builds for a particular machine, then we can modify the existing all builds view to group builds by machine. Based on this argument, I do not see a need for project and build hierarchy.

			[DAVID] My group understands that a "project" is a specific build directory. A project will most likely to have multiple builds, and things like the (a) machine, (b) layers, (c) conf variables, and so forth would be presumably set before the first target is selected and executed. It is also the case that the developer may decide to change any of those settings, including even the machine type, and then carry on with more builds. The use case would be for example is arch coverage testing of a particular set of kernel or user space packages.

			A host can then have multiple project directories (I most often work that way), for example when a developer truly wants to keep their development and/or debugging from not polluting other projects.

			Also, separate users should do their work in separate project directories, now that we are extending the model to cover that.

			There can then be multiple hosts, with all of their multiple projects, all channeling their Toaster data into a shared database. And with that you have a build farm.

			[RAVI] I see that the uniqueness of a project is the <hostname>://<build directory path>. The contents of the build directory can be changed and multiple builds can be kicked off from the same project with different configurations. If this the case, the project creation should also setup a build directory on the host.

			Looking at this use case, we should have toaster running on the host machine with a shared database. Or if we run a single toaster instance serving multiple projects (hosts and build directories), we should have a mechanism for the toaster machine to talk to the project machines. Am I missing something?

			[PAUL] Yes, but the project need not be tied to a specific host or location on disk. To my mind, the project is simply a configuration set - if you had a build farm the actual build could be run on any server within that farm and that side of it would be largely hidden from the user (until it needed to be visible, e.g. when you need to diagnose a host contamination issue.) This then means that the management of build directories can be independent, i.e. old build directories can be automatically deleted after a time period or when space runs low, but the project remains in the database for later use when needed.

			[DAVID] I am completely willing to accept the above definition of a "Toaster Project". The part that convinces me is that (a) while under active build and development it does map to a physical location, but (b) it can hold the data and presumably the "snapshot" artifacts of a physical build directory long after that actual directory has been deleted. That last part is a highly desired feature from the Wind River team, and was also brought up as a specific question at my Toaster presentation at the Yocto Project Summit.

			This implies that given the abstract nature of a Toaster "Project", it makes sense that not only can the developer share that "project" with another developer and Toaster instance (as per Belén design), but it makes sense that the original developer can make their own clones of that project when they for example want use as a reference basis for new projects with additional adjustments and/or for binding and running on a different host/directory.

			Here is what I would propose is the workflow.

			1) When a Toaster Project is created,

			* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

			* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

			* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory exists for an existing Project, then that is an error. This would allow us to re-connect to a deleted or unregistered YP build directories.

			* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

			2) For external hosts mounted via NFS:

			* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itself?

			3) For fully external hosts

			This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

			* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

			* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

	=== Hide builds from a project ===		=== Hide builds from a project ===

David Reyna: /* Discussion */

2014-04-18T18:10:09Z

Discussion

← Older revision		Revision as of 18:10, 18 April 2014
Line 88:		Line 88:

	[BELEN] This is a really good question. I need the technical crowd to establish what's possible / what we want to do. We can then design accordingly.		[BELEN] This is a really good question. I need the technical crowd to establish what's possible / what we want to do. We can then design accordingly.

			[DAVID] It is the case that we recommend all of our customers to add their content via layers, and specifically layer directories since they are most likely to be under continual and direct change and would not be placed into a formal repo, or more likely they would be managed by their SCM system as is and certainly not packed into a base git clone.

	=== Ability to identify / set project owner / user ===		=== Ability to identify / set project owner / user ===

← Older revision		Revision as of 14:34, 22 April 2014
Line 66:		Line 66:
	1) When a Toaster Project is created,		1) When a Toaster Project is created,

	* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.		* The "user" is the preset to the last "user", else if this is the first time it is defaulted to $USER. This is a required field.

	* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?		* The "email" is preset to the last email that matches "user", else it is empty. QUESTION: Is this a required field?

	* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory exists for an existing Project, then that is an error. This would allow us to re-connect to a deleted or unregistered YP build directories.		* The host:dir values are preset just as "poky/oe-init-build-env" would preset these values, in other words to the current machine and to "poky/../build" (unless redirected with the appropriate environment variables). If that directory (a) already exists and has a "conf/local.conf" file and (b) that directory exists for an existing Project, then that is an error. This would allow us to re-connect to a deleted or unregistered YP build directories.

	* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.		* We either have an explicit "Project Create" button, or we allow for the first build target command to insure that for new projects the project directory is created and the content preset by "poky/oe-init-build-env" before continuing.

	2) For external hosts mounted via NFS:		2) For external hosts mounted via NFS:

	* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itself?		* If an external host is mounted via a YP compatible NFS system, then that would be covered by the normal workflow above, using the local host and the respective NFS path. QUESTION: do we wish to identify the actual remove HOST ID in these circumstances, or just let the NFS path speak for itself?

	3) For fully external hosts		3) For fully external hosts
Line 82:		Line 82:
	This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.		This is the situation where a host remote to the Toaster, but is intended to be managed by this Toaster and use this Toaster's database.

	* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.		* This implies using some sort of rpc, probably via SSH, to execute the creation and build targets.

	* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.		* There would presumably be an installation of YP on that remote host, and that location would need to be known to Toaster. The installation could be as simple as an NFS mount that points to the local YP installation, which would provide a single source at the cost of NFS traffic.

	=== Hide builds from a project ===		=== Hide builds from a project ===