Toaster future release planning
From Yocto Project
Jump to navigationJump to search
2.2 Planning
Wish lists
Michael's
- Remove pseudo API cruft / template context to json response stuff
- Look at schema and rework where needed
- Consolidate used API into it's own view
- Finish toaster tables porting
- Create a toasterclient.py to interact with toaster to avoid need for 'command line builds'
- This would use toaster's API as the entry point to using bitbake
- It could be called by CI systems
- It could be used by auto builders etc
- Means that there are no two modes anywhere in Toaster
- Allows project configuration to be changed in toaster from the command line
Brian's
The Ordered Part
- Django 1.8 aka LTS
- Image Customization
- Testing and Stability
- One way to start/One code path
This is *not* ordered! This is more of a brain dump :)
- Collapse analysis and managed mode
- Working CI for each commit into toaster-next as well as for project peoples testME branches on poky-contrib:
- run django tests
- run selenium tests
- leverage sstate to make runs generally shorter
- remove git assumptions from code
- rationalize/simplify the configuration scripts
- if bitbake-memres comes out, interact with it gracefully (this is my preferred method of collapsing analysis and managed)
- fix the tmpdir/release build issue &/or discuss the need to support building old releases with a UI rather than limiting toaster to build the release it is part of. This is inherently fragile.
- ask bb to parse any layer added to a project and such as imported layers, and layers specified in the configuration file to give us recipe information.
- improve our build event following:
- note parse events
- update/retrieve ui so that we can tell something is happening before the first build event comes
- eliminate the remaining random tracebacks from try/except failures.
- allow configuration in addition to the addition/removal of layers/packages
- kernel config
- add user control/authentication
- update to Django 1.8. Always stay on Django LTS versions
- add layer update from layers.openembedded.org to the UI
- add Django tests for building e.g. we should be able to build core-image-sato from a django test and validate the db
- add mysql script to simplify setup with mysql/apache
- pull in new look and feel
- add django tests to validate all artifacts - if we say they can download it and it appears on a page anywhere, it should have a test
- improve docs of code on wiki. We have a nice start (ty E & M); I'd like to improve it.
- events - more info and list types supported
- what logic is in jscript , what is in python
- database schema definition and description esp interrelationships.
- list django tests, mostly so we can see what we are missing.
- build mode triggerred off of database entry, not autonomous process.
- add asynchronicity. I have noted a number of places where commands result in a sluggish ui. I'm nervous about this one.
- for example, starting a build that requires us to clone layers has a noticeable delay as compared to starting a local only project build.
- verify we can stop builds cleanly.
- get the rest of our logs into our local toaster dir
- allow us to run and stop 2 toasters on the same machine without them interfering with each other
- (2.2 or later for sure) discuss how to support multiple back end bb servers.
- probably will need to be able to add servers/remove servers/see server status/build state
- be able to see more details about build. For example, see the "jobs" being processed like you can in knotty
- revisit what variables we let people change. sstate_dir and dl_dir should certainly be allowed to change
- discuss way we do migrations. possibly change. thoughts:
- only support migrations from last release to current. (e.g. if you were running toaster jethro you could update to jethro++, but could not from fido to jethro)
- take migration from the startup script. it's only for releasetorelease.
- allow it to be in for master and ease of development w/in a release and pull at end as we do with moving releases.
- whatever we decide, add a test set for it so we know it works
- Be able to delete things from the ui
- old builds
- old projects
- imported layers we no longer want
- Be able to set from the ui a policy to delete all builds/recipes/packages that are older than XXX &/or be able to set maximum size on db and delete everything older than XXX when we near that size
- update the builds on the web page so we dont have to hit refresh to see the build progress.
Elliot's
- Make sure all files lint properly
- Include lint in CI builds
- Improve test coverage of all areas of the UI and back-end
- Clean up toasterui.py so it's easier to see what's going on in there
- Fix localhostbecontroller so it doesn't rely on what's in the log file when figuring out whether the bitbake server has started
- Provide a way to modify settings in settings.py without editing that file (e.g. with a local overlay configuration file)
- Move the logic, environment variables, directory setup and script calls from bin/toaster into Django commands: this would allow you to run toaster correctly, with all the required setup, without having to use bin/toaster; it would also mean that we could provide cleaner production setup instructions, as a user could run a series of Django commands (with their own parameters) to setup their environment; bin/toaster would then just execute a series of Django commands
Belén's
- Implement new custom theme (already in progress, aiming for end of M1: Dec 7th). Related bugs: 8417
- Finish image customisation. Related bugs: 8070, 8081, 8082, 8103, 8104, 8117, 8128, 8132, 8091
- Review interaction with layers: as it is, imposes heavy restrictions on users. Some of the questions I would like to consider:
- Should the import layer action be global instead of per project, so that you can reuse imported layers across projects?
- Should we allow users to override the release layer branch, setting a particular layer index layer to a different branch or commit?
- Should we allow users to set a different git repo for layer index layers?
- Should we allow users to override the bitbake revision set by the release in a project?
- Related bugs: 6640, 6701, 7574, 8426, 8429
- Access control (a simple one to start, with very basic permissions). Related bugs: 6233, 6234
- WIC integration (getting a step closer to the holy grail of 'click button' > 'image deployed in board')
- Fix the variable history (this is one of the features users have highly praised, and it is a bit broken in 2.0). Related bugs: 5811, 8488, 8190, 8189, 8188
- Get Toaster to collect build data from Jenkins builds. Once again, repeatedly requested by users. Related bugs: 7527
Ed's
- Merge analysis and managed modes
- Fix as much pending bugs as possible.
- Finish image customisation.
- Support Django LTS
- Increase test coverage. It would be good to set realistic goals, e.g. cover specific modules or increase coverage to 10% and follow the plan.
- Include production configuration into testing
- Cover views.py with tests and split it into modules. Currently it's huge and hard to maintain. I suspect there are a lot of unused APIs in there.
- Finish refactoring of buildcontroller code. This work is started in ed/toaster/bec and sitting there for quite long time.
- Bring patchset support to toaster patchwork.
- Add 'submitted upstream' and 'merged upstream' statuses to patchwork. Make the statuses updated automatically when patch is accepted to toaster-next or bitbake master.
- Clean up the code. Set realistic goals for pylint scores and follow them.
- Enhance Jenkins CI jobs to run all possible django, tts and pylint tests
Sujith's
- Get toaster working with non-git repos. Related bugs: 8456
- minor, but would be good to have :- Once user launches toaster, it would be nice to see the progress bar updated without user refreshing the page. Related bugs: 8328
- Cancellation of build. Related bugs: 6787
Would love to take up more tasks.
David's
- Finish the effort around Toaster "extensions", started by Farrell. This will enable Toaster customization for commercial partners, as per branding and
alternate pages (for extended project creation).
All bugs for 2.1+
ID | Milestone | Summary (164 tasks) ⇒ | Status | P |
---|---|---|---|---|
8651 | 2.1 | Toaster: Loading icon remains on screen and nothing appears in the project page | VERIFIED | Low |
7968 | Future | empty page states for no information in the database | RESOLVED | Low |
6670 | Future | all compatible recipes page should link to layer source recipe with the layer name | NEW | Low |
8326 | Future | Build time counts the time a build spent queued | NEW | Low |
9435 | Future | Toaster throws an exception for images that have files on the / level | NEW | Low |
8858 | Future | We should keep sorting state in our tables | NEW | Low |
8769 | Future | Integrating wic into Toaster | NEW | Low |
4402 | Future | Customise the summary of all the information available for one build + image recipe combination | NEW | Low |
10205 | Future | The layer typeahead shows the project release branch for non-git layers | NEW | Low |
6546 | 2.2 | Supporting use of local layers in Toaster | RESOLVED | Low |
8250 | 2.1 | In the 'new build' button, the project name link does not work | VERIFIED | Low |
7294 | Future | For QA Issues errors and warnings, toaster should recognize the error and provide suggestions to the user on how the error can be repaired | NEW | Low |
6978 | Future | For the local release in Toaster, show local path to layer directory instead of Git URL | NEW | Low |
8434 | Future | Toaster: package classes (rpm, ipk or deb) should be display in the build configuration page | NEW | Low |
9117 | Future | Allow "project builds" and "all builds" to be sorted by time and recipe name | NEW | Low |
8859 | Future | Provide a way of specifying the default sort direction in ToasterTable | NEW | Low |
7219 | Future | Show the file size of the license manifest when available for download in the build dashboard | NEW | Low |
8573 | 2.2 | [Toaster] the page heading doesn't change when search query returns no results | RESOLVED | Low |
10206 | Future | The add layer dependencies type ahead shows layers already in the list of dependencies | NEW | Low |
9856 | Future | Migrate remaining tables to ToasterTable | NEW | Low |
7297 | Future | need Toaster icon | VERIFIED | Low |
5811 | Future | the variable history should collect and display the variable value for each modification sequence | ACCEPTED | Low |
8807 | Future | local cached layers don't get the branch checked out | NEW | Low |
8685 | Future | Enter button doesn't apply your modification to bitbake variables | NEW | Low |
9118 | Future | Dead paths for meta-toaster-custom layer left behind in bblayers.conf | NEW | Low |
9831 | 2.2 | Pagination widget does not have the next and previous button for Packages, Tasks and Recipes pages | VERIFIED | Low |
7457 | Future | Split "Previous Build" tasks | NEW | Low |
8860 | Future | The 'clear search' icon in ToasterTable tables should clear any applied filter | NEW | Low |
8242 | 2.2 | Builds started from Toaster no longer show the 'queued' or 'checking out layers' stages | RESOLVED | Low |
8791 | 2.1 M3 | Software recipes table is not sort by any column by default | VERIFIED | Low |
8095 | Future | Table results should be sorted in alphabetical order and other secondary criterias. | NEW | Low |
9209 | Future | The unique name validation for custom images happens across projects | ACCEPTED | Low |
8577 | Future | Add a global UI widget to show progress of builds | NEW | Low |
10281 | Future | Configuration variables: I can set a value for standard shell environment variable http_proxy | NEW | Low |
6604 | Future | Clone a Toaster project | NEW | Low |
8425 | Future | Toaster: A warning should be displayed when you delete a layer dependency | NEW | Low |
9670 | Future | Toaster does not set a correct MIME type for tasks' logs files | NEW | Low |
7318 | 2.1 | Managed mode must use sessions to remember user search/orderby preferences | VERIFIED | Low |
8857 | Future | Display number of search / filter results consistently across tables | NEW | Low |
8036 | Future | Request to allow people to set pagination to all | NEW | Low |
9141 | Future | toaster when xmlrpc/cooker dies the web didn't tell the user about it | NEW | Low |
8546 | 2.1 | Toaster: add layer button from project configuration page remains sensitive if you select a layer with dependencies | VERIFIED | Low |
6778 | Future | Add "import layer" to layer suggestions drop down | NEW | Low |
9977 | Future | Time, CPU and Disk I/O tables have the wrong core columns | NEW | Low |
9839 | Future | The "clear search" button on the – "Add | Remove packages table" - does not clear | NEW | Low |
8865 | Future | The project "updated" field shows the last time a project was saved, not its last activity | NEW | Low |
8248 | Future | Toaster recipe page: missing empty state | NEW | Low |
8795 | 2.1 | In the build history information, layer names equal their directory name, not the layer name | VERIFIED | Medium |
8125 | 2.2 | Filter does not work as expected after clearing the search results | RESOLVED | Medium |
6954 | Future | Toaster Kernel Configuration Tool | RESOLVED | Medium |
4823 | Future | Provide generic, background information about layers | RESOLVED | Medium |
6897 | Future | Customise the Django admin interface | NEW | Medium |
4384 | Future | Visualising build time information | RESOLVED | Medium |
10526 | Future | Toaster: Add support for feed publishing | RESOLVED | Medium |
8426 | Future | Improve how we display layer dependencies | NEW | Medium |
7742 | Future | Toaster: Remove constraints when setting the layers checkout directory | NEW | Medium |
9914 | Future | For imported layers, remember the value of the last directory and the last Git repository | NEW | Medium |
9109 | Future | Apply sleep 0.5 to other database operations | NEW | Medium |
6166 | Future | Toaster collects no information about builds | VERIFIED | Medium |
9684 | Future | "Edit Columns" Button on the Compatible layers view is missing | NEW | Medium |
8896 | Future | Enable sorting by number of dependencies | NEW | Medium |
7334 | 2.1 | The 'machine' information is not searchable in the all builds, all projects and project builds tables | VERIFIED | Medium |
8220 | Future | We need to enable an exclusion list in toasterconf.json to drop layers from layers.openembedded.org | RESOLVED | Medium |
7198 | 2.2 | Hiding a sorted column does not revert to default sorting | RESOLVED | Medium |
5483 | Future | Extend Image creation to include workflow for security checking | RESOLVED | Medium |
8042 | 2.1 | Layerdetails/layers to show full dependency tree | VERIFIED | Medium |
6915 | Future | Toaster should handle ASSUME_PROVIDED | NEW | Medium |
9153 | Future | Enhance build performance data | NEW | Medium |
8572 | 2.1 | Json import error: Openembedded branches include local branch | VERIFIED | Medium |
7794 | Future | Design Toaster First Use | NEW | Medium |
6794 | Future | Add last commit information to Toaster | NEW | Medium |
4330 | Future | Visualising overall package size | RESOLVED | Medium |
9844 | 2.2 | Cancel build button does not work | VERIFIED | Medium |
8955 | Future | Recipe explorer | NEW | Medium |
8417 | Future | Merge new Bootstrap theme from yocto-webhob-design | RESOLVED | Medium |
9583 | Future | Refactor Toaster database schema | RESOLVED | Medium |
8866 | Future | Home-brew JSON encoding is ad hoc and incomplete | NEW | Medium |
5648 | Future | Add a field license_files to package information | ACCEPTED | Medium |
8806 | Future | Refactor localhostbecontroller | ACCEPTED | Medium |
8131 | 2.2 | All builds page "Errors"/"Warnings" filters doesn't work | RESOLVED | Medium |
4948 | 2.1 | create tests for Toaster API | VERIFIED | Medium |
9297 | Future | Toaster: Simplify building extensible SDK | NEW | Medium |
7977 | 2.1 | Toaster fails to start with Django 1.7 | VERIFIED | Medium |
6898 | Future | Design Toaster configuration interface | NEW | Medium |
4399 | Future | recipes: Visualising dependency relationships between recipes | RESOLVED | Medium |
10633 | Future | Add optional layers selection tool | NEW | Medium |
7747 | Future | Improve messaging when Toaster is launched with no releases set | ACCEPTED | Medium |
6701 | Future | Provide an option to select layers from different branches within a Toaster project | RESOLVED | Medium |
4285 | Future | tasks: Visualising relationships between tasks | RESOLVED | Medium |
9918 | Future | Use bitbakes fetcher code to grab layers, do checkouts, etc. | NEW | Medium |
6233 | Future | Design model for project owner / user | RESOLVED | Medium |
9812 | Future | Add hints to project configuration page that recipes may overwrite or modify settings during the build | NEW | Medium |
8899 | 2.1 | Change project release not working | VERIFIED | Medium |
8327 | 2.2 | Builds started from the 'all builds' page should show in the same page | RESOLVED | Medium |
7452 | Future | build mode path fields in the analysis pages should be a link to the specific details page | VERIFIED | Medium |
9485 | Future | Fix typeaheads for Bootstrap 3 | NEW | Medium |
8230 | Future | toaster upload failures on autobuilder | RESOLVED | Medium |
5538 | Future | Implement Backend API controller for buildbot | RESOLVED | Medium |
8790 | Future | Boostrapping package information for image customisation purposes | NEW | Medium |
8093 | Future | Release change doesn't check machine compatibility | RESOLVED | Medium |
6916 | Future | Toaster should turn QA check names in warnings / errors into links | NEW | Medium |
4624 | Future | Show the content of the run files in the task information pages | VERIFIED | Medium |
9164 | Future | The toaster-custom-images layer should not show like any other layer, and should not be removable | NEW | Medium |
6838 | Future | toaster must be able to submit error reports for failed builds from the interface | RESOLVED | Medium |
4331 | Future | Visualising single package size | NEW | Medium |
7596 | Future | variable type support in project configuration pages | NEW | Medium |
6603 | Future | Export / Import Toaster projects | NEW | Medium |
8959 | Future | Allow Toaster to run images on simulator, even a custom one | NEW | Medium |
8424 | Future | Toaster: Compatible recipes tab should display the machine compatible recipes | NEW | Medium |
9617 | Future | Create Toaster Docker production container | NEW | Medium |
8886 | 2.2 | Toaster thinks do_shared_workdir is a shared state task | RESOLVED | Medium |
8275 | 2.2 | Toaster: Suitable message and build box does not appear in build tab for a project with no builds | RESOLVED | Medium |
8189 | Future | Variable history shows full paths to files inside the /bitbake directory | NEW | Medium |
7038 | Future | For cached tasks, indicate in which URL the sstate object was found | NEW | Medium |
5192 | Future | non-existent URLs listed in the "Layer index URL" column | VERIFIED | Medium |
13245 | Future | Toaster visualization proposals | NEW | Medium |
9358 | 2.2 | [Test Case 1402] Test bitbake variables - IMAGE_FSTYPES | RESOLVED | Medium |
6900 | Future | Toaster should tell you when one of your local.conf variables has been overriden | RESOLVED | Medium |
4400 | Future | Visualising brought in by relationships between packages | RESOLVED | Medium |
8488 | Future | Collect variable information for the image scope | NEW | Medium |
7748 | Future | Provide a way to do a Toaster cleanup and start again if you get the initial set up wrong | NEW | Medium |
6712 | Future | Modify the default Bootstrap behaviour of tooltips and popovers triggered on hover | NEW | Medium |
4301 | Future | recipes: Visualising brought in by relationships between recipes | RESOLVED | Medium |
9969 | Future | Display target:task correctly on build dashboard (and elsewhere) | ACCEPTED | Medium |
6234 | Future | Implement model for project owner / user | NEW | Medium |
8366 | Future | Build the specific recipe version | NEW | Medium |
9518 | Future | Rationalise the directory structure generated by Toaster | NEW | Medium |
7234 | Future | warn the user if the project configuration settings aren't reflected in the build | NEW | Medium |
5560 | Future | toaster implement debug start option with data uploading | RESOLVED | Medium |
6953 | Future | Branding Support | NEW | Medium |
4819 | Future | webhob should be enabled through local.conf options | RESOLVED | Medium |
7945 | Future | Toaster startup script assumes it's in a git repository | RESOLVED | Medium |
6840 | Future | Seeing builds delta in Toaster | NEW | Medium |
4376 | Future | Visualising directory structure of built images | RESOLVED | Medium |
7727 | Future | toaster: show bbappend information for recipes | NEW | Medium |
9895 | Future | Provide UI to allow local layers to be published to layerindex. | NEW | Medium |
9005 | Future | Provide build cancellation functionality for command line builds | ACCEPTED | Medium |
8319 | Future | Improve the form provided by the 'new build' button | RESOLVED | Medium |
6095 | Future | Retrieve full build information independently of task execution | NEW | Medium |
9395 | Future | Cleanly support unicode throughout Toaster | RESOLVED | Medium |
8218 | Future | Provide a way to update layer information from the Toaster GUI | NEW | Medium |
7110 | Future | Builds of "world" are pretty ugly in toaster UI | NEW | Medium |
5255 | Future | Provide access to tasks executable output in Toaster | ACCEPTED | Medium |
9363 | 2.2 | When setting Bitbake variables the value reflected to toaster.conf doesn't respect the specified whitespace | RESOLVED | Medium |
8705 | Future | Store valid values for variables in the Toaster database | NEW | Medium |
6910 | Future | Use sticky headers for long tables | NEW | Medium |
4401 | Future | Collect aggregated usage data | RESOLVED | Medium |
7751 | Future | Toaster allows me to build when no machine is set in the default configuration | NEW | Medium |
4329 | Future | Visualising dependency and recommends relationships between packages | RESOLVED | Medium |
6239 | Future | Toaster utility commands for shared databases | VERIFIED | Medium |
8953 | 2.2 | Toaster ignores the subdirectory when cloning imported layers | RESOLVED | Medium |
8379 | 2.1 | "sorting" doesn't work well on "Recipe file" column | VERIFIED | Medium |
7527 | Future | Integrate Toaster with Jenkins | RESOLVED | Medium |
9519 | Future | Consolidate used API into it's own view | IN PROGRESS IMPLEMENTATION | Medium |
7235 | Future | warn the user about unexpected configuration changes in variable history | NEW | Medium |
5564 | Future | toaster document REST APIs dynamically using Swagger | VERIFIED | Medium |
9266 | 2.1 M4 | Image builds aren't identified as images | VERIFIED | Medium+ |
9385 | 2.1 M4 | After a project's initial image build of sato, building a custom minimal never shows packages | VERIFIED | Medium+ |
7574 | 2.2 | The project release change process deletes imported layers without warning | RESOLVED | Medium+ |
7914 | Future | toaster should start remote builds through a single script | RESOLVED | Medium+ |
8026 | 2.1 | Toaster start up directories incorrect | VERIFIED | Medium+ |
8901 | 2.1 M3 | Split package groups from all other packages in build information | VERIFIED | Medium+ |
8895 | 2.1 M3 | Lsupdates throw error using mysql | VERIFIED | Medium+ |
8659 | 2.1 | Loading the openembedded-core toasterconf.json file shows 2 openembedded-core layers | RESOLVED | Undecided |
Toaster Enhancements for 2.2
ID | Assignee | E | Summary (17 tasks) ⇒ | P | Requester | QA | Milestone | Status | Resolution | Whiteboard |
---|---|---|---|---|---|---|---|---|---|---|
6546 | Belen Barros Pena | 1 | Supporting use of local layers in Toaster | Low | Belen Barros Pena | 2.2 | RESOLVED | OBSOLETE | GUI design available | |
8573 | Belen Barros Pena | [Toaster] the page heading doesn't change when search query returns no results | Low | Cristina Agurida | Cristina Agurida | 2.2 | RESOLVED | DUPLICATE | GUI design pending | |
9002 | Elliot Smith | 6 | Move directory scan logic out of toaster.bbclass into buildinfohelper | Low | Elliot Smith | Libertad | 2.2 M2 | RESOLVED | FIXED | |
8456 | Michael Wood | Add feature to add layers to Toaster from external source | Medium | Michael Wood | Libertad | 2.2 | RESOLVED | FIXED | ||
8678 | Libertad | 1 | Testing coverage of non-poky Toaster setup | Medium | Belen Barros Pena | 2.2 | RESOLVED | FIXED | ||
9582 | Michael Wood | 14 | Replace toasterconf.json with Django fixtures | Medium | Belen Barros Pena | Libertad | 2.2 | RESOLVED | FIXED | |
7744 | Belen Barros Pena | 3 | Improve messaging in empty states when Toaster does not have any layer information | Medium | Belen Barros Pena | 2.2 M4 | RESOLVED | WONTFIX | GUI design pending | |
9184 | Michael Wood | 2 | Cannot delete layer from toaster database | Medium+ | Henry Bruce | Libertad | 2.2 | RESOLVED | FIXED | |
8563 | Elliot Smith | 5 | Improve detection and presentation of build artifacts | Medium+ | Belen Barros Pena | Libertad | 2.2 M2 | RESOLVED | FIXED | GUI desing available |
9023 | Elliot Smith | 4 | Improve the content of the "other artifacts" section | Medium+ | Belen Barros Pena | Libertad | 2.2 M2 | RESOLVED | FIXED | GUI design available |
9631 | Elliot Smith | 10 | add state Transitions to progress bar as well as build progress. | Medium+ | brian avery | Libertad | 2.2 M3 | RESOLVED | FIXED | GUI design available |
9911 | Sujith H | Import a layer that points to a directory instead of a Git repository | Medium+ | Belen Barros Pena | Libertad | 2.2 M3 | RESOLVED | FIXED | GUI design available | |
9912 | Sujith H | Build a layer that points to a directory instead of a Git repository | Medium+ | Belen Barros Pena | Libertad | 2.2 M3 | RESOLVED | FIXED | ||
9913 | Sujith H | Switch imported layers between directories and Git repositories | Medium+ | Belen Barros Pena | Libertad | 2.2 M3 | RESOLVED | FIXED | GUI design available | |
9925 | Sujith H | Support building non-Git layers with Toaster | Medium+ | Belen Barros Pena | Libertad | 2.2 M3 | RESOLVED | FIXED | ||
6238 | Michael Wood | 2 | delete projects/builds through the toaster UI | Medium+ | David Reyna | Libertad | 2.2 M4 | RESOLVED | FIXED | GUI design available |
8132 | Michael Wood | 4 | Implement deleting custom images | Medium+ | Belen Barros Pena | Libertad | 2.2 M4 | RESOLVED | FIXED | GUI design available |