Toaster future release planning
From Yocto Project
Jump to navigationJump to search
2.1 Planning
Wish lists
Michael's
- Finish image-customisation 8070, 8081, 8082, 8103, 8104, 8117, 8128, 8132, 8091
- Remove pseudo API cruft / template context to json response stuff
- Consolidate used API into it's own view
- Finish toaster tables porting
- Get selenium test wrapper
- Create a toasterclient.py to interact with toaster to avoid need for 'command line builds'
- This would use toaster's API as the entry point to using bitbake
- It could be called by CI systems
- It could be used by auto builders etc
- Means that there are no two modes anywhere in Toaster
- Allows project configuration to be changed in toaster from the command line
Brian's
This is *not* ordered! This is more of a brain dump :)
- Collapse analysis and managed mode
- Working CI for each commit into toaster-next as well as for project peoples testME branches on poky-contrib:
- run django tests
- run selenium tests
- leverage sstate to make runs generally shorter
- remove git assumptions from code
- rationalize/simplify the configuration scripts
- if bitbake-memres comes out, interact with it gracefully (this is my preferred method of collapsing analysis and managed)
- fix the tmpdir/release build issue &/or discuss the need to support building old releases with a UI rather than limiting toaster to build the release it is part of. This is inherently fragile.
- ask bb to parse any layer added to a project and such as imported layers, and layers specified in the configuration file to give us recipe information.
- improve our build event following:
- note parse events
- update/retrieve ui so that we can tell something is happening before the first build event comes
- eliminate the remaining random tracebacks from try/except failures.
- allow configuration in addition to the addition/removal of layers/packages
- kernel config
- add user control/authentication
- update to Django 1.8. Always stay on Django LTS versions
- add layer update from layers.openembedded.org to the UI
- add Django tests for building e.g. we should be able to build core-image-sato from a django test and validate the db
- add mysql script to simplify setup with mysql/apache
- pull in new look and feel
- add django tests to validate all artifacts - if we say they can download it and it appears on a page anywhere, it should have a test
- improve docs of code on wiki. We have a nice start (ty E & M); I'd like to improve it.
- events - more info and list types supported
- what logic is in jscript , what is in python
- database schema definition and description esp interrelationships.
- list django tests, mostly so we can see what we are missing.
- build mode triggerred off of database entry, not autonomous process.
- add asynchronicity. I have noted a number of places where commands result in a sluggish ui. I'm nervous about this one.
- for example, starting a build that requires us to clone layers has a noticeable delay as compared to starting a local only project build.
- verify we can stop builds cleanly.
- get the rest of our logs into our local toaster dir
- allow us to run and stop 2 toasters on the same machine without them interfering with each other
- (2.2 or later for sure) discuss how to support multiple back end bb servers.
- probably will need to be able to add servers/remove servers/see server status/build state
- be able to see more details about build. For example, see the "jobs" being processed like you can in knotty
- revisit what variables we let people change. sstate_dir and dl_dir should certainly be allowed to change
- discuss way we do migrations. possibly change. thoughts:
- only support migrations from last release to current. (e.g. if you were running toaster jethro you could update to jethro++, but could not from fido to jethro)
- take migration from the startup script. it's only for releasetorelease.
- allow it to be in for master and ease of development w/in a release and pull at end as we do with moving releases.
- whatever we decide, add a test set for it so we know it works
- Be able to delete things from the ui
- old builds
- old projects
- imported layers we no longer want
- Be able to set from the ui a policy to delete all builds/recipes/packages that are older than XXX &/or be able to set maximum size on db and delete everything older than XXX when we near that size
- update the builds on the web page so we dont have to hit refresh to see the build progress.
Elliot's
- Make sure all files lint properly
- Include lint in CI builds
- Improve test coverage of all areas of the UI and back-end
- Clean up toasterui.py so it's easier to see what's going on in there
- Fix localhostbecontroller so it doesn't rely on what's in the log file when figuring out whether the bitbake server has started
- Provide a way to modify settings in settings.py without editing that file (e.g. with a local overlay configuration file)
- Move the logic, environment variables, directory setup and script calls from bin/toaster into Django commands: this would allow you to run toaster correctly, with all the required setup, without having to use bin/toaster; it would also mean that we could provide cleaner production setup instructions, as a user could run a series of Django commands (with their own parameters) to setup their environment; bin/toaster would then just execute a series of Django commands
Belén's
- Implement new custom theme (already in progress, aiming for end of M1: Dec 7th). Related bugs: 8417
- Finish image customisation. Related bugs: 8070, 8081, 8082, 8103, 8104, 8117, 8128, 8132, 8091
- Review interaction with layers: as it is, imposes heavy restrictions on users. Some of the questions I would like to consider:
- Should the import layer action be global instead of per project, so that you can reuse imported layers across projects?
- Should we allow users to override the release layer branch, setting a particular layer index layer to a different branch or commit?
- Should we allow users to set a different git repo for layer index layers?
- Should we allow users to override the bitbake revision set by the release in a project?
- Related bugs: 6640, 6701, 7574, 8426, 8429
- Access control (a simple one to start, with very basic permissions). Related bugs: 6233, 6234
- WIC integration (getting a step closer to the holy grail of 'click button' > 'image deployed in board')
- Fix the variable history (this is one of the features users have highly praised, and it is a bit broken in 2.0). Related bugs: 5811, 8488, 8190, 8189, 8188
- Get Toaster to collect build data from Jenkins builds. Once again, repeatedly requested by users. Related bugs: 7527
Ed's
- Merge analysis and managed modes
- Fix as much pending bugs as possible.
- Finish image customisation.
- Support Django LTS
- Increase test coverage. It would be good to set realistic goals, e.g. cover specific modules or increase coverage to 10% and follow the plan.
- Include production configuration into testing
- Cover views.py with tests and split it into modules. Currently it's huge and hard to maintain. I suspect there are a lot of unused APIs in there.
- Finish refactoring of buildcontroller code. This work is started in ed/toaster/bec and sitting there for quite long time.
- Bring patchset support to toaster patchwork.
- Add 'submitted upstream' and 'merged upstream' statuses to patchwork. Make the statuses updated automatically when patch is accepted to toaster-next or bitbake master.
- Clean up the code. Set realistic goals for pylint scores and follow them.
- Enhance Jenkins CI jobs to run all possible django, tts and pylint tests
All bugs for 2.1+
| ID | Milestone | Summary (188 tasks) ⇒ | Status | P |
|---|---|---|---|---|
| 8669 | 2.1 M1 | Wrong password encoding syntax error in "bash -c" | VERIFIED | High |
| 8056 | 2.1 M3 | Image customisation front end implementation | VERIFIED | High |
| 8364 | 2.1 M1 | Upgrade Toaster to use Django 1.8 | VERIFIED | High |
| 9101 | 2.1 M4 | Dependencies dialog when adding packages to an image does not always work | VERIFIED | High |
| 8910 | 2.1 M2 | Toaster builds fail at the rootfs stage | VERIFIED | High |
| 9240 | 2.1 M4 | builds don't start in production setup mode | VERIFIED | High |
| 9856 | Future | Migrate remaining tables to ToasterTable | NEW | Low |
| 8577 | Future | Add a global UI widget to show progress of builds | NEW | Low |
| 8095 | Future | Table results should be sorted in alphabetical order and other secondary criterias. | NEW | Low |
| 9117 | Future | Allow "project builds" and "all builds" to be sorted by time and recipe name | NEW | Low |
| 8807 | Future | local cached layers don't get the branch checked out | NEW | Low |
| 6604 | Future | Clone a Toaster project | NEW | Low |
| 7161 | 2.1 | Inconsistent presentation of builds with more than one target | VERIFIED | Low |
| 4402 | Future | Customise the summary of all the information available for one build + image recipe combination | NEW | Low |
| 8695 | 2.1 | No default sorting applied in recipes and machines layer details tables | VERIFIED | Low |
| 8858 | Future | We should keep sorting state in our tables | NEW | Low |
| 9326 | 2.1 M4 | 'type the recipe you want to build' help says use semicolon but shows colon | VERIFIED | Low |
| 10205 | Future | The layer typeahead shows the project release branch for non-git layers | NEW | Low |
| 8860 | Future | The 'clear search' icon in ToasterTable tables should clear any applied filter | NEW | Low |
| 9435 | Future | Toaster throws an exception for images that have files on the / level | NEW | Low |
| 6778 | Future | Add "import layer" to layer suggestions drop down | NEW | Low |
| 7294 | Future | For QA Issues errors and warnings, toaster should recognize the error and provide suggestions to the user on how the error can be repaired | NEW | Low |
| 10281 | Future | Configuration variables: I can set a value for standard shell environment variable http_proxy | NEW | Low |
| 8769 | Future | Integrating wic into Toaster | NEW | Low |
| 9141 | Future | toaster when xmlrpc/cooker dies the web didn't tell the user about it | NEW | Low |
| 8476 | 2.1 | mysql port must be specified in settings.py even though marked as optional | VERIFIED | Low |
| 8036 | Future | Request to allow people to set pagination to all | NEW | Low |
| 8248 | Future | Toaster recipe page: missing empty state | NEW | Low |
| 7318 | 2.1 | Managed mode must use sessions to remember user search/orderby preferences | VERIFIED | Low |
| 8511 | 2.1 | Import layer page, "Add layer" button remains enabled when layer text input is empty | VERIFIED | Low |
| 8791 | 2.1 M3 | Software recipes table is not sort by any column by default | VERIFIED | Low |
| 9839 | Future | The "clear search" button on the – "Add | Remove packages table" - does not clear | NEW | Low |
| 9209 | Future | The unique name validation for custom images happens across projects | ACCEPTED | Low |
| 6978 | Future | For the local release in Toaster, show local path to layer directory instead of Git URL | NEW | Low |
| 8651 | 2.1 | Toaster: Loading icon remains on screen and nothing appears in the project page | VERIFIED | Low |
| 8421 | 2.1 | Configure default columns for the image image recipes and software recipes tables | VERIFIED | Low |
| 8685 | Future | Enter button doesn't apply your modification to bitbake variables | NEW | Low |
| 9118 | Future | Dead paths for meta-toaster-custom layer left behind in bblayers.conf | NEW | Low |
| 8857 | Future | Display number of search / filter results consistently across tables | NEW | Low |
| 8425 | Future | Toaster: A warning should be displayed when you delete a layer dependency | NEW | Low |
| 6670 | Future | all compatible recipes page should link to layer source recipe with the layer name | NEW | Low |
| 7219 | Future | Show the file size of the license manifest when available for download in the build dashboard | NEW | Low |
| 9977 | Future | Time, CPU and Disk I/O tables have the wrong core columns | NEW | Low |
| 9122 | 2.1 M3 | Name your custom image text box not showing properly | VERIFIED | Low |
| 8859 | Future | Provide a way of specifying the default sort direction in ToasterTable | NEW | Low |
| 8434 | Future | Toaster: package classes (rpm, ipk or deb) should be display in the build configuration page | NEW | Low |
| 10206 | Future | The add layer dependencies type ahead shows layers already in the list of dependencies | NEW | Low |
| 8865 | Future | The project "updated" field shows the last time a project was saved, not its last activity | NEW | Low |
| 8247 | 2.1 | The build form in the project page does not interact with adding / removing layers | VERIFIED | Low |
| 7297 | Future | need Toaster icon | VERIFIED | Low |
| 8250 | 2.1 | In the 'new build' button, the project name link does not work | VERIFIED | Low |
| 9670 | Future | Toaster does not set a correct MIME type for tasks' logs files | NEW | Low |
| 9165 | 2.1 M4 | Replace 'delete' with 'remove' in the context of project layer manipulation | VERIFIED | Low |
| 8326 | Future | Build time counts the time a build spent queued | NEW | Low |
| 7457 | Future | Split "Previous Build" tasks | NEW | Low |
| 5811 | Future | the variable history should collect and display the variable value for each modification sequence | ACCEPTED | Low |
| 8905 | 2.1 M3 | Toaster: add layer button from project configuration page remains sensitive | VERIFIED | Low |
| 8546 | 2.1 | Toaster: add layer button from project configuration page remains sensitive if you select a layer with dependencies | VERIFIED | Low |
| 8792 | 2.1 M3 | Errors column and project column does not disappear when unchecked from edit columns menu | VERIFIED | Medium |
| 9206 | 2.1 M4 | Building custom images based on custom images fails | VERIFIED | Medium |
| 8366 | Future | Build the specific recipe version | NEW | Medium |
| 6166 | Future | Toaster collects no information about builds | VERIFIED | Medium |
| 6953 | Future | Branding Support | NEW | Medium |
| 8796 | 2.1 M3 | Remove all manual transaction management | VERIFIED | Medium |
| 9229 | 2.1 M4 | Layers from layer index not available when using local Yocto release | VERIFIED | Medium |
| 8379 | 2.1 | "sorting" doesn't work well on "Recipe file" column | VERIFIED | Medium |
| 7742 | Future | Toaster: Remove constraints when setting the layers checkout directory | NEW | Medium |
| 6239 | Future | Toaster utility commands for shared databases | VERIFIED | Medium |
| 9111 | 2.1 M4 | Custom images should not be listed in the image recipes table | VERIFIED | Medium |
| 7038 | Future | For cached tasks, indicate in which URL the sstate object was found | NEW | Medium |
| 9914 | Future | For imported layers, remember the value of the last directory and the last Git repository | NEW | Medium |
| 8424 | Future | Toaster: Compatible recipes tab should display the machine compatible recipes | NEW | Medium |
| 7748 | Future | Provide a way to do a Toaster cleanup and start again if you get the initial set up wrong | NEW | Medium |
| 8944 | 2.1 M3 | Compatible metadata is shared across projects using the local release | VERIFIED | Medium |
| 9969 | Future | Display target:task correctly on build dashboard (and elsewhere) | ACCEPTED | Medium |
| 8426 | Future | Improve how we display layer dependencies | NEW | Medium |
| 7794 | Future | Design Toaster First Use | NEW | Medium |
| 6706 | Future | Include maintainers list for layers | CLOSED | Medium |
| 7234 | Future | warn the user if the project configuration settings aren't reflected in the build | NEW | Medium |
| 4948 | 2.1 | create tests for Toaster API | VERIFIED | Medium |
| 8449 | 2.1 | Toaster: add layer button from project configuration page remains sensitive after you delete from text box | VERIFIED | Medium |
| 7977 | 2.1 | Toaster fails to start with Django 1.7 | VERIFIED | Medium |
| 8218 | Future | Provide a way to update layer information from the Toaster GUI | NEW | Medium |
| 5255 | Future | Provide access to tasks executable output in Toaster | ACCEPTED | Medium |
| 8959 | Future | Allow Toaster to run images on simulator, even a custom one | NEW | Medium |
| 8866 | Future | Home-brew JSON encoding is ad hoc and incomplete | NEW | Medium |
| 9518 | Future | Rationalise the directory structure generated by Toaster | NEW | Medium |
| 6794 | Future | Add last commit information to Toaster | NEW | Medium |
| 9156 | 2.1 M4 | Adding a package that was a dependency doesn't always look right | VERIFIED | Medium |
| 5539 | Future | Design Project Management REST API | VERIFIED | Medium |
| 13245 | Future | Toaster visualization proposals | NEW | Medium |
| 8776 | 2.1 | We need to catch the error: WARNING: Could not connect to server at 0.0.0.0:46572 (Could not register UI event handler) ERROR: Could not connect to server 0.0.0.0:46572 : Could not register UI event handler | VERIFIED | Medium |
| 9153 | Future | Enhance build performance data | NEW | Medium |
| 8896 | Future | Enable sorting by number of dependencies | NEW | Medium |
| 9617 | Future | Create Toaster Docker production container | NEW | Medium |
| 6897 | Future | Customise the Django admin interface | NEW | Medium |
| 8782 | 2.1 | toastermain settings.py hardcoded filecache location | VERIFIED | Medium |
| 9164 | Future | The toaster-custom-images layer should not show like any other layer, and should not be removable | NEW | Medium |
| 7452 | Future | build mode path fields in the analysis pages should be a link to the specific details page | VERIFIED | Medium |
| 5648 | Future | Add a field license_files to package information | ACCEPTED | Medium |
| 9058 | 2.1 | Bad project MACHINE setting in conf/toaster.conf causes command line builds to fail | VERIFIED | Medium |
| 6910 | Future | Use sticky headers for long tables | NEW | Medium |
| 9684 | Future | "Edit Columns" Button on the Compatible layers view is missing | NEW | Medium |
| 8528 | 2.1 | Toaster does not recognize image recipes when it builds them on local project | VERIFIED | Medium |
| 9200 | 2.1 M4 | we are disabling file based git urls in imported (actually all) layers | VERIFIED | Medium |
| 7596 | Future | variable type support in project configuration pages | NEW | Medium |
| 6095 | Future | Retrieve full build information independently of task execution | NEW | Medium |
| 6916 | Future | Toaster should turn QA check names in warnings / errors into links | NEW | Medium |
| 8572 | 2.1 | Json import error: Openembedded branches include local branch | VERIFIED | Medium |
| 8795 | 2.1 | In the build history information, layer names equal their directory name, not the layer name | VERIFIED | Medium |
| 7727 | Future | toaster: show bbappend information for recipes | NEW | Medium |
| 6234 | Future | Implement model for project owner / user | NEW | Medium |
| 9109 | Future | Apply sleep 0.5 to other database operations | NEW | Medium |
| 9895 | Future | Provide UI to allow local layers to be published to layerindex. | NEW | Medium |
| 8806 | Future | Refactor localhostbecontroller | ACCEPTED | Medium |
| 7747 | Future | Improve messaging when Toaster is launched with no releases set | ACCEPTED | Medium |
| 6603 | Future | Export / Import Toaster projects | NEW | Medium |
| 7110 | Future | Builds of "world" are pretty ugly in toaster UI | NEW | Medium |
| 4331 | Future | Visualising single package size | NEW | Medium |
| 9918 | Future | Use bitbakes fetcher code to grab layers, do checkouts, etc. | NEW | Medium |
| 9297 | Future | Toaster: Simplify building extensible SDK | NEW | Medium |
| 7751 | Future | Toaster allows me to build when no machine is set in the default configuration | NEW | Medium |
| 8148 | 2.1 | Sorting the content of the Layer/Machines table doesn't work | VERIFIED | Medium |
| 4624 | Future | Show the content of the run files in the task information pages | VERIFIED | Medium |
| 8955 | Future | Recipe explorer | NEW | Medium |
| 8705 | Future | Store valid values for variables in the Toaster database | NEW | Medium |
| 6712 | Future | Modify the default Bootstrap behaviour of tooltips and popovers triggered on hover | NEW | Medium |
| 8189 | Future | Variable history shows full paths to files inside the /bitbake directory | NEW | Medium |
| 7235 | Future | warn the user about unexpected configuration changes in variable history | NEW | Medium |
| 5192 | Future | non-existent URLs listed in the "Layer index URL" column | VERIFIED | Medium |
| 8958 | 2.1 M4 | Toaster does not build with the Jethro release | VERIFIED | Medium |
| 9137 | 2.1 M4 | Improve handling of builds without package data | VERIFIED | Medium |
| 9485 | Future | Fix typeaheads for Bootstrap 3 | NEW | Medium |
| 8450 | 2.1 | Toaster: Build more recipes from Most build recipes fails | VERIFIED | Medium |
| 6787 | 2.1 | Cancelling builds at any point in the build process | VERIFIED | Medium |
| 5385 | Future | Create users data model. | VERIFIED | Medium |
| 9005 | Future | Provide build cancellation functionality for command line builds | ACCEPTED | Medium |
| 10633 | Future | Add optional layers selection tool | NEW | Medium |
| 8775 | 2.1 | check to see if port is in use early and exit | VERIFIED | Medium |
| 9142 | 2.1 M4 | ca-certificates have non ascii characters in filenames which causes toaster to be unhappy | VERIFIED | Medium |
| 9519 | Future | Consolidate used API into it's own view | IN PROGRESS IMPLEMENTATION | Medium |
| 8488 | Future | Collect variable information for the image scope | NEW | Medium |
| 8042 | 2.1 | Layerdetails/layers to show full dependency tree | VERIFIED | Medium |
| 6840 | Future | Seeing builds delta in Toaster | NEW | Medium |
| 8781 | 2.1 | Cannot build lsb images | VERIFIED | Medium |
| 9163 | 2.1 M4 | Add reverse dependencies column to the image details pages | VERIFIED | Medium |
| 7334 | 2.1 | The 'machine' information is not searchable in the all builds, all projects and project builds tables | VERIFIED | Medium |
| 5564 | Future | toaster document REST APIs dynamically using Swagger | VERIFIED | Medium |
| 8899 | 2.1 | Change project release not working | VERIFIED | Medium |
| 8527 | 2.1 | Sorting in layer details page jumps back to the default tab (layer details) | VERIFIED | Medium |
| 6898 | Future | Design Toaster configuration interface | NEW | Medium |
| 8790 | Future | Boostrapping package information for image customisation purposes | NEW | Medium |
| 9087 | 2.1 M4 | Toaster hangs when you start a build specifying an invalid task | VERIFIED | Medium |
| 6915 | Future | Toaster should handle ASSUME_PROVIDED | NEW | Medium |
| 9812 | Future | Add hints to project configuration page that recipes may overwrite or modify settings during the build | NEW | Medium |
| 7603 | 2.1 M3 | Toaster is not storing in its database information about SDK artifacts | VERIFIED | Medium+ |
| 9108 | 2.1 M4 | The image recipe details page show the list of packages built, instead of the list of packages installed | VERIFIED | Medium+ |
| 8917 | 2.1 M3 | Remove the ability to change the project release | VERIFIED | Medium+ |
| 4315 | Future | Create a local bitbake run manager | VERIFIED | Medium+ |
| 8927 | 2.1 M3 | Custom image package list does not show packages installed via IMAGE_LINGUAS | VERIFIED | Medium+ |
| 8104 | 2.1 M3 | Implement building a custom image | VERIFIED | Medium+ |
| 9266 | 2.1 M4 | Image builds aren't identified as images | VERIFIED | Medium+ |
| 8128 | 2.1 M3 | Implement downloading custom image recipe files | VERIFIED | Medium+ |
| 9121 | 2.1 M4 | Remove reverse dependencies when removing packages from custom images | VERIFIED | Medium+ |
| 8154 | 2.1 M1 | Remove the master release from fido | VERIFIED | Medium+ |
| 8956 | 2.1 M4 | Toaster doesn't display build artifacts | VERIFIED | Medium+ |
| 8725 | 2.1 M3 | Impossible to access the command line builds project page (project 1) | VERIFIED | Medium+ |
| 9123 | 2.1 M4 | Build history pages are missing the image customisation links | VERIFIED | Medium+ |
| 9021 | 2.1 M3 | Toaster merges builds started from its web UI with builds started from CLI | VERIFIED | Medium+ |
| 8279 | 2.1 | Provide a single way of starting Toaster | VERIFIED | Medium+ |
| 8901 | 2.1 M3 | Split package groups from all other packages in build information | VERIFIED | Medium+ |
| 8081 | 2.1 M1 | Implement the 'new custom image' page | VERIFIED | Medium+ |
| 8091 | 2.1 M3 | Implement the 'custom images' page | VERIFIED | Medium+ |
| 8373 | 2.1 M1 | Toaster in analysis mode downloads the wrong cooker logs | VERIFIED | Medium+ |
| 8925 | 2.1 M3 | Custom image package list does not show kernel-related packages | VERIFIED | Medium+ |
| 8103 | 2.1 M3 | Implement creating a custom image | VERIFIED | Medium+ |
| 9116 | 2.1 M3 | migrations fail to apply on mysql | VERIFIED | Medium+ |
| 8933 | 2.1 M3 | Image customisation package list does not show packages installed via dependencies | VERIFIED | Medium+ |
| 8117 | 2.1 M1 | Implementing adding and removing packages to / from custom images | VERIFIED | Medium+ |
| 9385 | 2.1 M4 | After a project's initial image build of sato, building a custom minimal never shows packages | VERIFIED | Medium+ |
| 7880 | 2.1 M4 | Toaster uses the same /tmp dir across releases | VERIFIED | Medium+ |
| 8738 | 2.1 M3 | Build and Projects tables not using ToasterTables | VERIFIED | Medium+ |
| 8026 | 2.1 | Toaster start up directories incorrect | VERIFIED | Medium+ |
| 8895 | 2.1 M3 | Lsupdates throw error using mysql | VERIFIED | Medium+ |
| 9051 | 2.1 M3 | IDs missing from toaster main page | VERIFIED | Medium+ |
| 9154 | 2.1 M4 | The custom image details page needs a custom 'no search results' page | VERIFIED | Medium+ |
| 8070 | 2.1 M3 | Implement image recipe details page | VERIFIED | Medium+ |
| 8082 | 2.1 M3 | Implement the 'custom image details' page | VERIFIED | Medium+ |
