Future Directions
Potential Future Project Directions
The Yocto Project TSC has been giving thought to the “big picture” areas of the project which could potentially be developed in the future. These are listed below, where possible the TSC has tried to indicate what these topics are, why they are important, their current status, developments that are happening and what else we believe might need to be done. If there are other areas which anyone believes should be on this list, please raise it with the TSC. The list is in approximate priority order from the TSC’s perspective although in reality, resource constraints can impact ability to make progress on any topic area, or allow it to happen if it is sponsored by a contributor.
In generating this list the TSC is considering many factors and are considering this from a project wide perspective. We realise individual contributors and members have different priorities and the priority will also be heavily influenced by where resources are available to help, be it from the community or from members. Assigning a priority is hard as the TSC can have limited impact on many topics as it has no resources to assign.
LTS
LTS proposal already published and acted upon, see LTS.
Security
There are two areas the project has been asked to define or provide:
- PSiRT
- This is a group of individuals who have the task of monitoring, responding to and logging vulnerabilities for the Project.
- By using various input sources, issues are evaluated and the appropriate actions are taken.
- A scoring matrix will need to be defined
- A set of tools will need to be defined to assist in this effort
- The scope of work needs to be defined.
- Layers covered.
- Do we do scoring ourselves?
- How to prioritize high profile issues
- Level of triaging
- Level 1 - in pkg list
- Level 2 - pkg versions
- Level 3 - Source check
Resourcing estimate: This task would need a team of people on an ongoing basis to run the monitoring and analysis. Multiple people would be needed to provide backup coverage and the time commitment could be a few hours a week ( Depends on scope). There is also a question of the underlying tools which would need a maintainer with availability of a few hours per month for maintenance and development time for the development of tools which could be around 2-4 weeks/year for a 'medium' skilled developer with some domain knowledge. The tools are isolated with few inter-dependencies and project background knowledge isn't essential. The system the tools run on will need to be under maintenance.
'Example time estimate: Only Poky investigating Pre-scored CVE's to Level 2, 10 CVEs/hour
- Vulnerability mitigation
- This is the process of removing a security vulnerability either by a patch, configuration change or workaround.
- Assumption - a patch exists upstream
- If a POC exists, us it for validation else whatever QA tests exist (testimage)
- Skillset
- Familiarity in the programming language the package is written in.
- Has sent patches to YP/OE community mailing lists
Resourcing estimate: This task would involve a team of people but is hard to predict due to the "bursty" nature of the activity and the range of areas mitigations could be required within. Usually the project would wait for fixes upstream and then it becomes a question of upgrading to those or backporting the changes to the current development branch and the actively maintained stable series (Are the LTS branches included?). Ideally a multi-person team is needed with someone able to cover a few hours per week.(Under estimated ).
'Example time estimate: 1 CVE/hour/branch ( includes backporting, building and some level of validation)
Current status: We have CVE reports being sent regularly to the yocto-security list for OE-Core but not other layers. We do not have any dashboard or visualisation of the numbers. We do not have any teams established for monitoring or mitigation, nor developers actively working on the tools.
Layer setup/configuration
Currently each user of the project is responsible for setting up layers and stub configuration files themselves. This could include downloading bitbake and metadata layers and then generating configuration for the user, then later also support updating some of the components as appropriate.
There are multiple approaches out there (combo-layer, git submodules, repo, WindRiver setuptool, YP autobuilder, KAS tool from Siemens. other shell/python scripts like bitbake-layers). Each has pros/cons. This fragmented approach isn’t good for the project, it would be better if we had a unified approach. One tool probably won’t fit all but there are too many right now. This task would be to investigate the area and come up with a proposal/path to improving things.
Resourcing Estimate: The task is a single developer task which will probably take around 2 months work even for a highly skilled domain expert. Skills needed are background knowledge on how we end up where we are and community discussion/liaison ability to figure out the correct path forward on this topic. Ongoing maintenance could be 1day/month for bug fixing.
YP Compatibility
The current YP compatibility programme has stalled. From a technical perspective there are minor gremlins but we believe it's the right direction.
The issues have been:
- problems with the webform
- nobody to resolve open bugs with the process
- process improvements needed (ensure the tools are run consistently with reproducible results)
- current testing process is too manual
- not being marketed enough
In the view of the TSC, the project is currently missing out on opportunities here, both to attract new members and to improve the quality of the layers/metadata. It is suffering from lacking someone to own and drive the process.
Developer Tools and Processes
Currently we have devtool, eSDK and unmaintained eclipse integration. Need a plan to improve eSDK, have a build which can migrate between “sdk” and “full” build modes. Also need plans for new IDE integration, particularly Microsoft Visual Studio Code which appears to be the modern IDE to target. There is a long standing bug for a “lock/unlock” bitbake command to move sections of recipes between the two modes which would be one stepping stone to completing this work and improving eSDK usability. Tools to show the locked status of recipes would also help, as would better tools to debug signature differences.
Secondly, adding better layer setup/configuration tooling would remove the need to handle this in eSDK, removing a fragile part of the current codebase [see separate item above].
Usability
The TSC discussed usability which is a wide-ranging topic. We believe that several of the specific topic areas proposed previously actually work towards the goal of improving project usability. For example, the patch submission process would benefit new developers who are more familiar with a “github” style patch submission process. Another significant change would be the layer setup process which would help with another subsection of usability. There is also a community aspect to usability through different mediums such as stack overflow, twitch, OEDEM/OEDAM, devdays and all the other programmes the project participates in or runs.
A critical area in the TSC’s opinion is the enhancement and redevelopment of eSDK and tools like devtool (and creation of new tools alongside it) as the original idea was good but the implementation has issues which nobody is currently working on. This area has not been scoped out by the TSC as a specific proposal as yet but is a well-known area of the project that needs development.
Usability has so many different focuses that different people will prioritise different components of this and have different ideas. It also depends on the skills of any resource available as to the pace any given area can move forward.
QA automation
It has become very clear that automated testing is our only realistic way forward for the project. We’ve made massive improvements to the infrastructure both in hardware (through a refresh) and in software (yocto-autobuilder2/yocto-autobuilder-helper/resulttool).
We need to:
- raise awareness of what automation capability we already have
- have more people looking at the test results
- Work in improving test results (many ptests failures currently)
- Improve release test comparison so its reliable/useful
- encourage more people to submit test results against a release
- finish new testing documentation manual
Example Test Report:
Armin will use meta-security as an example? Become a talk?
Code Submission process
The project uses what is considered to be a fairly ‘old school’ approach to code submissions. The older project members are comfortable with this and it has advantages (like peer review). There are more modern approaches which may encourage contributions from a wider audience.
The OE community is trying an experiment with gitlab with some specific layers where the maintainer is willing to experiment to understand what works well and what doesn’t, we’re likely to await the results of that to decide how to proceed:
Using GitLab for OE/Yocto layers
Binary Distro
There are a subset of users of the project who need “binary package feeds” similar to a conventional Linux distro. They don’t want users to have to build things themselves, simply download and use them. Users look at Debian/Ubuntu and other distros and don’t understand why YP doesn’t do this. The difference between a source distro and a binary distro is lost on them.
The challenges:
- finding a configuration which all these users could agree on to have a feed which would work for multiple consumers
- deciding which binaries should be supported (on which architectures and target optimizations)
- maintaining a potentially large set of such binaries
- testing upgrade paths between releases
There is a large resource commitment needed, even if much automation can be made to try and make this possible. If the project itself hosting a binary distro isn’t viable, having tools and adding testing for the processes needed to do this would be extremely valuable.
Software Update
Currently the only official update mechanism YP supports is through conventional package management (rpm/ipk/deb). There are other better alternatives for some situations such as ‘A’/’B’ updaters or update mechanisms with less granularity than packages. Mender and Resin have commercial solutions, there is also swupdate and OSTree in particular. These provide whole system updates in one, or larger groupings of updates than just packages.
Open questions are whether any of this should be core technology and whether anything in core should change to better support this.
At the very least we need to document the options users have for software update.
What the project likely needs to do in this space is provide infrastructure/prototypes and automated tests of the building blocks and processes people would need to do any of the above.
Modern Programming Language Runtimes
We see patterns where there are new language runtimes such as rust, go, nodejs and to some extent, perl and python where there are large numbers of dependencies which don’t conform to our ideas of larger units of software with specific releases/versions that are well defined by recipes. We need to consider strategies at the architecture level to work better with these.
AI/ML
We also should consider strategies at the architecture level to cover and unify AI/ML usage in/by the project.
Other future topics
Other areas which need development:
- Support for multiple toolchains (clang or arch specific e.g. intel or arm compilers) as not all recipes build with standard gcc any longer
