Hob2 (Hob v2)

From Yocto Project
Jump to navigationJump to search

This page holds the history for Hob v2 design and implementation.

Requirements

See slides attached. File:HOBV2-plan-0.2.pdf

Related Bugs

Discussion 1 (Lianhao, Ke, Dongxiao and Shane)

In order to get the accurate dependencies of the packages to be installed into a rootfs image, the hob v2 works in a 2-stages way: In stage 1, the users select the recipes which would be built to generate packages; In stage 2, the users select the packages which would be installed into the final rootfs image. The hob v2 will present the UI to the users in the form of a step-by-step wizard. Here is the main workflow:

Step 1: Users are asked to specify which layers(directories) should be included.

- TP1: Hob would check for the validity of the file conf/layer.conf to make sure the user input is a valid layer.

[P1][D5]http://bugzilla.pokylinux.org/show_bug.cgi?id=1742

Step 2: Users are asked to select various configurations, i.e. machine, distro, package format, image output type, etc.

- TP2: bitbake backend will then parse the configuration files and all the available recipes based on user's configuration.

[P1][D5]http://bugzilla.pokylinux.org/show_bug.cgi?id=1743

Step 3: Users are asked to select the recipes which will be built later to generate the packages. In this step, the users may select the recipes from scratch or select the recipes from the pre-defined base image templates, e.g. core-image-sato, core-image-minimal, etc.

- TP3: Hob frontend needs the information of all the available recipes, e.g. PN, license, description, section, etc.

[P1][D5]http://bugzilla.pokylinux.org/show_bug.cgi?id=1745


- TP4: Hob frontend needs to know the build dependencies between the recipes. Say recipe A has a build dependency on recipe B, if the users select recipe A, recipe B should also be automatically selected. Without deselecting recipe A, the users can NOT deselect recipe B only. If the users deselect recipe A, the recipe B will still remain selected.

[P1][D10]http://bugzilla.pokylinux.org/show_bug.cgi?id=1747

- TP5: bitbake backend needs to be able to generate the recipe dependencies quick enough so the Hob frontend UI won't wait too long.

[P1][TBD]http://bugzilla.pokylinux.org/show_bug.cgi?id=1749

Step 4: After the users select all the recipes they want to build and click "Next", bitbake will build them. This may take quite a long time if those recipes have never been built or if there is no sstate. During that time, the building log will be displayed in the hob UI along with the progress bar. The users may choose to cancel the build if they want.

- TP6: A progress bar should tell the users how much left for the building.

[P1][D3]http://bugzilla.pokylinux.org/show_bug.cgi?id=1751

Step 5: After all recipes are built successfully, the users will be asked to choose which packages they want to install into the final rootfs images. The available packages can be sorted by Name/Section/Size/Selected or not. The users may also be given the opportunity to configure other image options if appropriate, i.e. how much free space will be added into the final images, do we need prelink or mklibs on the final images, etc. Some packages are by default selected to be installed based on the base image template the users have chosen in step3.

- TP7: New items, i.e. package installed size, will be added into the current pkgdata.

[P1][D3]http://bugzilla.pokylinux.org/show_bug.cgi?id=1748

- TP8: Need to develop a new bitbake task to collect all the pkgdata information and send these information back to Hob UI frontend

[P1][D5]http://bugzilla.pokylinux.org/show_bug.cgi?id=1750

- TP9: Hob UI frontend will use the information generated in TP8 to build a reference count based tree model about the packages dependencies. This tree model helps the users to figure out what actually will be installed into the final rootfs images.

[P1][D15]http://bugzilla.pokylinux.org/show_bug.cgi?id=1752

- TP10: Hob UI needs to write a temporary recipe and reparse that recipes to build the final images.

[P1][D3]http://bugzilla.pokylinux.org/show_bug.cgi?id=1746

Other TPs: - TP11: users should be able to load/save their configurations, choices of recipes and/or packages.

[P1][D3]http://bugzilla.pokylinux.org/show_bug.cgi?id=1744

Open Issues(OI): - OI1: For TP5, we need to investigate whether the bitbake backend can generate the accurate recipe build dependencies using only cache data and taskData, without calling prepare_run_queue() which would take quite a long time because it will generate all the task dependencies. If not, we need to figure out a way to accelerate that process. One way is to write another special prepare_run_queue so it only process the information the hob concerns. But RP is concerned that if we have 2 kind of prepare_run_queue, we need to make sure the dependencies generated by each of them are exactly the same.

- OI2: How to handle the build dependencies of virtual targets. E.g. many recipes have build dependencies on virtual/libc, and we have both eglibc and uclibc to provide virtual/libc. Eglibc will be the default options but what if the user wants uclibc instead of eglibc. One possible way may be to treat this as part of the configuration PREPERRED_PROVIDER, and have the bitbake backend reparse the recipes based on the new configurations, but this way may take some time due to the reparsing.

[P1][TBD]http://bugzilla.pokylinux.org/show_bug.cgi?id=1753

Discussion 2 (Lianhao, Josh and Jessica)

1. It's feasible to save the UI configurations in the client side, not in the server side's conf/local.conf (Bug #1441), but there is one thing need needs special attention: the cache validity. Currently, the cache validity is decided by comparing the file modification time(See code in Cache:__init__() in cache.py). If we change the configuration in the data store but not in the server side's configuration file, that means we have to design a new way to decide the cache validity.

2. Currently, the server commands getVariable and setVariable only apply to the base configuration data store. It doesn't apply to the data store of a specific recipe. It's feasible to extend those commands, in which case we don't need to create new recipe for stage 2 building, but again need to pay attention to cache validity. Josh prefers the way to create new recipe for stage 2 building because it is simpler.

3. About OI1, Josh thinks taskData itself is not enough to generate accurate build dependencies. Josh is ok with NOT providing build dependencies in stage 1, but RP thinks we need to show that build dependencies .

AR: confirm whether we could generate correct build dependencies based on taskData only. If the answer is no, then we need to see how the lengthy operation of prepare_runqueue will influence the user experience. Is it ok with users if they're given a progress bar indicating what is going on during that period of time(more than 20 seconds I guess including the taskData, prepare_runqueue and pre-process the returned data)? If it's not acceptable, we may need to persuade RP not showing the build dependencies in stage1, at least for 1.2 version, unless we have other ways to get the dependencies in a short time.

4. About OI2, Josh thinks it's the right way to use PREPERRED_PROVIDERS as part of the configurations for multiple provider situation.

5. Josh thinks the current metadata information in the recipe may not be sufficient for hob use. He prefers to add more metadata into the recipe. I raised the question of whether the community is willing to accept it, Josh thinks we should do that work for the recipes we own, just like the distro tracking data we added.

6. Jessica/Josh mentioned that we should keep extensibility in our minds during HobV2 design, since we already got comment from community saying that they want to add more configurations in stage 2 building. I mentioned about the idea of having a configuration file for the configurable items in HobV2. Josh mentioned there is already some kind of mechanism in bitbake which would be a good start point for us(See commit 4921fda5b3a18c797acd267f2b1179ac032cd42).

7. Jessica thinks we kind of need to demo the Hobv2 skeleton UI at the end of M1(Bug #1764). There will be a person in UK helping us with the UX design, Dave suggest this "artist" thought might change our engineer mindset.

8. Jessica suggests that we need to do the split between client/server in 1.2, in an evolutionary way that Hobv2 uses the modal of client/server split, while other UI may continue to use the modal of client/server on the same machine.

UI Skelecton

<That is based on our previous discussions (esp. discussion 1) and just for review, the UI design could be changed and polished, some details could be determined later>

1. Here is the main window with menus.

Main window.png

“New Build” is to reset the process to the initial state to start a new build process. “Open Template” is to load a hob-specific file saved last time, which includes setting, recipes, and packages. “Close Template” is just to close the current process, discard any change to the template file. “Save Template” is to save the changes of the template file. “Save Template As” is to save the changes to another template file. A file save dialog is shown. “Exit” is to exit the program.

The items under “Window” are to switch to any step from any step.

The “view” items under “Tools” are to view different files and logs. The “Open” items are to open different directories. It is worth noting that “Open target directory” is useful after the target image is made. "One click to build and make" is that the user doesn't want to click "Next >" one by one, but load his/her template, use all settings in the template file and click this to make the target image. “Deploy live image to USB drive” is to make a live image based on the image made. "kernel config" is to "menu config" in the kernel for kernel customization. (TBD)

The menus under “Help” are to show help hints, files and copyright information.

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Hob2_(Hob_v2)&oldid=4039"