FutureMigrationGuide: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
No edit summary
 
(50 intermediate revisions by 6 users not shown)
Line 5: Line 5:
|}
|}


=2.4 Migration Guide items=
=3.0 Migration Guide items=
* ROOTFS_PKGMANAGE_BOOTSTRAP is gone, and any references to it should be removed.
* The meta-yocto directory has been removed - since the 2.1 release when meta-yocto was renamed to meta-poky, the meta-yocto subdirectory remained to avoid breaking existing configurations.
* maintainers.inc, the file which tracks maintainers with primary responsibility for each recipe in OE-Core has been moved from meta-poky to OE-Core, i.e. from meta-poky/conf/distro/include to meta/conf/distro/include.


== Memory resident mode ==
== Init system selection ==


Bitbake's previous "memory resident mode" has been replaced by a persistent mode which is available in bitbake's default operation. As such, you only need to set BB_SERVER_TIMEOUT to a timeout (in seconds) and BitBake's server will stay resident for that amount of time between bitbake invocations. The oe-init-build-env-memres script has been removed as a separate environment setup script is no longer needed.
Changing the init system manager previously required setting a number of different variables; this can now be accomplished by setting the INIT_MANAGER variable and corresponding include files (conf/distro/include/init-manager-*.conf); include files are provided for four values - "none", "sysvinit", "systemd" and "mdev-busybox". The default value "none" for INIT_MANAGER should allow your current settings to continue working, but it is advisable to set INIT_MANAGER explicitly.
 
== LSB support removed ==
 
LSB as a standard isn't current, and isn't well suited to embedded applications. Support may be continued in a separate layer if needed but at present it has been removed from the core.
 
Following on from this change, the poky-lsb derivative distribution configuration that was also used for testing alternative configurations has been replaced with a poky-altcfg distribution that has LSB parts removed.
 
== Removed recipes ==
 
* core-image-lsb-dev: part of removed LSB support
* core-image-lsb: part of removed LSB support
* core-image-lsb-sdk: part of removed LSB support
* cve-check-tool: functionally replaced by cve-update-db recipe and cve-check class
* eglinfo: no longer maintained. eglinfo from mesa-demos is an adequate and maintained alternative.
* gcc-8.3: version 8.3 removed (replaced by 9.2)
* gnome-themes-standard: only needed by gtk+ 2.x which has been removed.
* gtk+: GTK+ 2 is obsolete (replaced by gtk+3)
* irda-utils: obsolete - IrDA support has been removed from the Linux kernel in version 4.17 and later.
* libnewt-python: libnewt Python support merged into main libnewt recipe
* libsdl: replaced by newer libsdl2
* libx11-diet: obsolete
* libxx86dga: remove obsolete client libary
* libxx86misc: remove this now redundant library
* linux-yocto: version 5.0 removed (5.2 / 4.19 present)
* lsbinitscripts: part of removed LSB support
* lsb: part of removed LSB support
* lsbtest: part of removed LSB support
* openssl10: replaced by newer openssl 1.1
* packagegroup-core-lsb: part of removed LSB support
* python-nose: remove the python 2.x version of the recipe
* python-numpy: remove the python 2.x version of the recipe
* python-scons: remove the python 2.x version of the recipe
* source-highlight: no longer needed
* stress: replaced by stress-ng
* vulkan: split into vulkan-loader, vulkan-headers and vulkan-tools
* weston-conf: functionality moved to weston-init


== Packaging changes ==
== Packaging changes ==


* python3: the main "python3" package now brings in all of the standard Python 3 distribution rather than just a subset, matching the typical expected behaviour based on traditional Linux distributions. If you wish to install a subset of Python 3, specify python-core plus one or more of the individual packages that are still produced.
* The epiphany browser has been dropped from packagegroup-self-hosted as it has not been needed inside build-appliance-image for a long time and was causing resource problems.
* python3: bz2.py, lzma.py and _compression.py have beem moved from the python3-misc package to the python3-compression package.
* libcap-ng python support has been moved to a separate libcap-ng-python recipe to streamline the build process when the python bindings aren't needed
* binutils: libbfd is now packaged in a separate "libbfd" package in order to save space when some tools (notably perf) are installed, since they only need libbfd rather than all of binutils.
* libdrm now packages the amdgpu.ids file into a separate libdrm-amdgpu package
* util-linux: su is now packaged in a separate "util-linux-su" package (only built when "pam" is in DISTRO_FEATURES), since su is normally provided by shadow and thus the util-linux version shouldn't be installed unless it is needed. The main util-linux package RDEPENDS on this package when "pam" is in DISTRO_FEATURES.
* python3: the runpy module is now in the python3-core package as it is required to support the common "python3 -m" command usage.
* util-linux: switch_root is now packaged in a separate "util-linux-switch-root" package for small initramfs images which don't need the whole util-linux package (nor the busybox binary which is much larger than switch_root). The main util-linux package RRECOMMENDS this package.
* distcc now provides separate distcc-client and distcc-server packages as typically one or the other are needed, rather than both.
* util-linux: ionice is now packaged in a separate "util-linux-ionice" package for convenience. The main util-linux package RRECOMMENDS this package.
* python*-setuptools recipes now separately package the pkg_resources module in a python-pkg-resources / python3-pkg-resources package as the module is useful independent of the rest of the setuptools package. The main python-setuptools / python3-setuptools package depends on this new package so you should only need to update dependencies unless you want to take advantage of the increased granularity.
* initscripts: sushell is now package in a separate "initscripts-sushell" package to allow systemd to pull it in when selinux is enabled without needing to pull in the entire initscripts package. The main initscripts package RDEPENDS on this package when "selinux" is in DISTRO_FEATURES.
* glib-2.0 now RRECOMMENDS shared-mime-info, as large portions of GIO are not that useful without the MIME database. This can be removed using BAD_RECOMMENDATIONS if shared-mime-info is too large and isn't required.
* The Go standard runtime has been split out from the main go recipe into a separate "go-runtime" recipe.


== Removed recipes ==
== CVE checking ==
 
cve-check-tool has been functionally replaced by a new cve-update-db recipe and functionality built into the cve-check class. The result uses NVD JSON data feeds rather than the deprecated XML feeds that cve-check-tool was using, supports CVSSv3 scoring and makes other improvements.
 
Additionally, the CVE_CHECK_CVE_WHITELIST variable has been replaced by CVE_CHECK_WHITELIST.
 
== Bitbake changes ==
 
* addtask statements now properly validate dependent tasks - previously an invalid task here would be silently ignored, now it will generate a warning.
* Other invalid addtask and deltask usages will now trigger warnings (multiple target tasks arguments with addtask / deltask, multiple before/after clauses)
* The "multiconfig" prefix is now shortened to "mc"; "multiconfig" will continue to work, however it may be removed in a future release.
* bitbake -g no longer generates a recipe-depends.dot file as the contents (a reprocessed version of task-depends.dot) were confusing.
* The bb.build.FuncFailed exception, previously raised by bb.build.exec_func() when certain other exceptions have occurred, has been removed. The real underlying exceptions will be raised instead. If you have calls to bb.build.exec_func() in custom classes or tinfoil-using scripts, any references to bb.build.FuncFailed should be cleaned up.
* Additionally, the bb.build.exec_func() no longer accepts the "pythonexception" parameter - it now always raises exceptions. Remove this argument in any calls to bb.build.exec_func() in custom classes or scripts.
* The BB_SETSCENE_VERIFY_FUNCTION2 is no longer used - in the unlikely event that you have any references to it, they should be removed.
* The RunQueueExecuteScenequeue and RunQueueExecuteTasks events have been removed since setscene tasks are now executed as part of the normal runqueue. Any event handling code in custom classes or scripts that handles these two events will need to be updated.
* The arguments passed to functions used with BB_HASHCHECK_FUNCTION have changed. If you are using own custom hash check function, see http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=40a5e193c4ba45c928fccd899415ea56b5417725 for details.
* Task specifications in BB_TASKDEPDATA and class implementations used in signature generator classes now use "<fn>:<task>" everywhere rather than the "." delimiter that was being used in some places. This makes it consistent with everywhere else in the code. Custom signature generator classes and code that reads BB_TASKDEPDATA will need to be updated to use ':' as a separator rather than '.'.


The following recipes have been removed in the 2.4 release:
== Sanity checks ==


* acpitests: recipe not being maintained
* SRC_URI is now checked for usage of two problematic items:
* autogen-native: with grub no longer requiring it, there's nothing else in oe-core or meta-oe that does.
*# "${PN}" - instead "${BPN}" should be used to avoid problems when PN is prefixed or suffixed e.g. with multiconfig.
* bdwgc: moved to meta-oe as nothing in OpenEmbedded-Core needs it in any longer.
*# github archive tarballs - these are not guaranteed to be stable and thus it is likely that they will be refreshed and thus the SRC_URI checksums will fail to apply. It is recommended that you fetch either an official release tarball or a specific revision from the actual git repository instead.
* byacc - was only needed by rpm 5.x and has been moved to meta-oe
*:Either one of these will now trigger a warning by default. If you wish to disable this check, remove src-uri-bad from WARN_QA.
* gcc (5.4): drop 5.4 series in favour of 6.3 / 7.2
* The file-rdeps runtime dependency check no longer expands RDEPENDS recursively as there is no mechanism to ensure they can be fully computed, and thus races sometimes resulted in errors showing up and sometimes not. Thus you may now see errors for missing runtime dependencies that were previously satisfied recursively e.g. package A contains a shell script starting with #!/bin/bash but has no dependency on bash, but depends on package B that does depend on bash. Add the missing dependency / dependencies to resolve the warning.  
* gnome-common: deprecated upstream and no longer needed
* Setting DEPENDS_${PN} anywhere (typically in a recipe) will now trigger an error - this is invalid, as DEPENDS is not a package-specific variable (unlike RDEPENDS). Set DEPENDS instead.
* go-bootstrap-native: remove recipe as go 1.9 can do its own bootstrapping
* systemd is currently known not to work well with the musl C library as upstream only officially supports linking it with glibc. Thus, a warning will now be shown when building systemd in conjunction with musl.
* guile: was only needed by autogen-native and remake, so no longer needed
* libclass-isa-perl: was previously needed for LSB 4, no longer needed
* libdumpvalue-perl: was previously needed for LSB 4, no longer needed
* libenv-perl: was previously needed for LSB 4, no longer needed
* libfile-checktree-perl: was previously needed for LSB 4, no longer needed
* libi18n-collate-perl: was previously needed for LSB 4, no longer needed
* libiconv: this was only needed for uclibc which was removed in the previous release - glibc and musl have their own implementations. meta-mingw still needs it so it has been moved there.
* libpng12: only previously needed for LSB; current libpng is 1.6.x.
* libpod-plainer-perl: was previously needed for LSB 4, no longer needed
* linux-yocto (4.1): removed in favour of 4.4, 4.9, 4.10 and 4.12
* mailx: only previously needed for LSB compatibility, with upstream being defunct now for a long time.
* mesa (git version only) - has gone stale with respect to the release version
* ofono (git version only) - has gone stale with respect to the release version
* portmap: obsolete - superseded by rpcbind.
* python3-pygpgme: old and unmaintained; was previously required only by dnf but that has now switched to official gpgme python bindings.
* python-async: removed in favour of python 3 version
* python-gitdb: removed in favour of python 3 version
* python-git: removed in favour of python 3 version
* python-mako: removed in favour of python 3 version
* python-pexpect: removed in favour of python 3 version
* python-ptyprocess: removed in favour of python 3 version
* python-pycurl: nothing is using this in OpenEmbedded-Core (or meta-oe)
* python-six: removed in favour of python 3 version
* python-smmap: removed in favour of python 3 version
* remake: using remake as the provider of virtual/make has been broken for some time. No longer needed in OpenEmbedded-Core.


== Other changes (yet to be categorised / completed) ==
== Miscellaneous ==


* The buildhistory class now makes a single commit per build rather than one per subdirectory in the repository (assuming commits are enabled with BUILDHISTORY_COMMIT = "1" as is typical). The earlier behaviour was intended to make it easier to   see just the changes for a particular subdirectory, but git can filter that itself when viewing - just specify that subdirectory as the last parameter on the git show / git diff command line and that's all you will see.
* The gnome class has been removed, as it did not do very much anymore. Recipes that previously inherited this class should be updated to do the following:
* x86-base.inc: Don't add live to IMAGE_FSTYPES, default instead
inherit gnomebase gtk-icon-cache gconf mime
* image_types.bbclass: get rid of IMAGE_DEPENDS
* The meta/recipes-kernel/linux/linux-dtb.inc file has been removed. It was previously deprecated (in favour of setting KERNEL_DEVICETREE in any kernel recipe) and only produced a warning. Remove any include/require statements pointing to this file.
* bitbake: cooker: Use multiple BuildStarted events for multiconfig
* TARGET_CFLAGS, TARGET_CPPFLAGS, TARGET_CXXFLAGS, and TARGET_LDFLAGS are no longer exported to the external environment. This has not required changes to any recipes, but if for some reason the software being built by one of your recipes is expecting these variables to be set then it will now fail, and you will need to either export the variable(s) in the recipe or change the scripts so that that isn't necessary.
* security_flags.inc: Delete pinnings for SECURITY_NO_PIE_CFLAGS
* The host distro identifier as used in NATIVELSBSTRING (when not using uninative) and SANITY_TESTED_DISTROS is now always lowercased even if it does not contain a version number.
* bitbake: tinfoil: add a parse_recipes() function
* In the base-files recipe, writing the hostname into /etc/hosts and /etc/hostname is now done within the main do_install function rather than do_install_basefilesissue, so that do_install_basefilesissue can be more easily overridden without having to duplicate the hostname functionality. If you have done the latter (e.g. in a base-files bbappend) then you should remove it from your customised do_install_basefilesissue function.
* bsp/help: include a warning showing deprecation of create-layer plugin
* The wic --expand command-line option now uses commas to separate key:value pairs rather than hyphens, as was documented in the command-line help. Any scripts or commands where you use wic --expand with multiple key:value pairs will need to be updated.
* image: Convert vmdk/vdi/qcow2 to strict CONVERSION_CMD types
* UEFI image variable settings have been moved from various places to a central conf/image-uefi.conf. However, this should not influence any existing configuration as the meta/conf/image-uefi.conf in the core metadata sets defaults that can be overridden in the same manner as before.
* insane: remove unsafe-references-in-scripts check
* conf/distro/include/world-broken.inc has been removed - where we need to disable certain recipes when using the musl C library, individual recipes now have COMPATIBLE_HOST_libc-musl set with a comment explaining why.
* OpenSSL 1.1 has been introduced, but the default is still 1.0.x via PREFERRED_VERSION due to remaining compatibility issues with other software. "openssl10" has been added to PROVIDES as a marker for recipes that still definitely depend on OpenSSL 1.0.
* bitbake: cooker: Change to consistent prefile/postfile handling
* insane.bbclass: Warn if ${COREBASE}/LICENSE is used
* kernel: Move Device Tree support to kernel.bbclass
* README.qemu: Add from meta-yocto as it belongs in core
* meta-yocto: Restructure and tidy up READMEs

Latest revision as of 21:32, 22 October 2019

This page should be used to keep track of items which should be added to the migration guide in the YP Reference Manual. It's a living document for the life of a development cycle and by M4 should represent a significant proportion of items that are worthy of note in the migration guide.

Once the migration guide for a release has been completed the contents of this page should be cleared out.

3.0 Migration Guide items

Init system selection

Changing the init system manager previously required setting a number of different variables; this can now be accomplished by setting the INIT_MANAGER variable and corresponding include files (conf/distro/include/init-manager-*.conf); include files are provided for four values - "none", "sysvinit", "systemd" and "mdev-busybox". The default value "none" for INIT_MANAGER should allow your current settings to continue working, but it is advisable to set INIT_MANAGER explicitly.

LSB support removed

LSB as a standard isn't current, and isn't well suited to embedded applications. Support may be continued in a separate layer if needed but at present it has been removed from the core.

Following on from this change, the poky-lsb derivative distribution configuration that was also used for testing alternative configurations has been replaced with a poky-altcfg distribution that has LSB parts removed.

Removed recipes

  • core-image-lsb-dev: part of removed LSB support
  • core-image-lsb: part of removed LSB support
  • core-image-lsb-sdk: part of removed LSB support
  • cve-check-tool: functionally replaced by cve-update-db recipe and cve-check class
  • eglinfo: no longer maintained. eglinfo from mesa-demos is an adequate and maintained alternative.
  • gcc-8.3: version 8.3 removed (replaced by 9.2)
  • gnome-themes-standard: only needed by gtk+ 2.x which has been removed.
  • gtk+: GTK+ 2 is obsolete (replaced by gtk+3)
  • irda-utils: obsolete - IrDA support has been removed from the Linux kernel in version 4.17 and later.
  • libnewt-python: libnewt Python support merged into main libnewt recipe
  • libsdl: replaced by newer libsdl2
  • libx11-diet: obsolete
  • libxx86dga: remove obsolete client libary
  • libxx86misc: remove this now redundant library
  • linux-yocto: version 5.0 removed (5.2 / 4.19 present)
  • lsbinitscripts: part of removed LSB support
  • lsb: part of removed LSB support
  • lsbtest: part of removed LSB support
  • openssl10: replaced by newer openssl 1.1
  • packagegroup-core-lsb: part of removed LSB support
  • python-nose: remove the python 2.x version of the recipe
  • python-numpy: remove the python 2.x version of the recipe
  • python-scons: remove the python 2.x version of the recipe
  • source-highlight: no longer needed
  • stress: replaced by stress-ng
  • vulkan: split into vulkan-loader, vulkan-headers and vulkan-tools
  • weston-conf: functionality moved to weston-init

Packaging changes

  • The epiphany browser has been dropped from packagegroup-self-hosted as it has not been needed inside build-appliance-image for a long time and was causing resource problems.
  • libcap-ng python support has been moved to a separate libcap-ng-python recipe to streamline the build process when the python bindings aren't needed
  • libdrm now packages the amdgpu.ids file into a separate libdrm-amdgpu package
  • python3: the runpy module is now in the python3-core package as it is required to support the common "python3 -m" command usage.
  • distcc now provides separate distcc-client and distcc-server packages as typically one or the other are needed, rather than both.
  • python*-setuptools recipes now separately package the pkg_resources module in a python-pkg-resources / python3-pkg-resources package as the module is useful independent of the rest of the setuptools package. The main python-setuptools / python3-setuptools package depends on this new package so you should only need to update dependencies unless you want to take advantage of the increased granularity.

CVE checking

cve-check-tool has been functionally replaced by a new cve-update-db recipe and functionality built into the cve-check class. The result uses NVD JSON data feeds rather than the deprecated XML feeds that cve-check-tool was using, supports CVSSv3 scoring and makes other improvements.

Additionally, the CVE_CHECK_CVE_WHITELIST variable has been replaced by CVE_CHECK_WHITELIST.

Bitbake changes

  • addtask statements now properly validate dependent tasks - previously an invalid task here would be silently ignored, now it will generate a warning.
  • Other invalid addtask and deltask usages will now trigger warnings (multiple target tasks arguments with addtask / deltask, multiple before/after clauses)
  • The "multiconfig" prefix is now shortened to "mc"; "multiconfig" will continue to work, however it may be removed in a future release.
  • bitbake -g no longer generates a recipe-depends.dot file as the contents (a reprocessed version of task-depends.dot) were confusing.
  • The bb.build.FuncFailed exception, previously raised by bb.build.exec_func() when certain other exceptions have occurred, has been removed. The real underlying exceptions will be raised instead. If you have calls to bb.build.exec_func() in custom classes or tinfoil-using scripts, any references to bb.build.FuncFailed should be cleaned up.
  • Additionally, the bb.build.exec_func() no longer accepts the "pythonexception" parameter - it now always raises exceptions. Remove this argument in any calls to bb.build.exec_func() in custom classes or scripts.
  • The BB_SETSCENE_VERIFY_FUNCTION2 is no longer used - in the unlikely event that you have any references to it, they should be removed.
  • The RunQueueExecuteScenequeue and RunQueueExecuteTasks events have been removed since setscene tasks are now executed as part of the normal runqueue. Any event handling code in custom classes or scripts that handles these two events will need to be updated.
  • The arguments passed to functions used with BB_HASHCHECK_FUNCTION have changed. If you are using own custom hash check function, see http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=40a5e193c4ba45c928fccd899415ea56b5417725 for details.
  • Task specifications in BB_TASKDEPDATA and class implementations used in signature generator classes now use "<fn>:<task>" everywhere rather than the "." delimiter that was being used in some places. This makes it consistent with everywhere else in the code. Custom signature generator classes and code that reads BB_TASKDEPDATA will need to be updated to use ':' as a separator rather than '.'.

Sanity checks

  • SRC_URI is now checked for usage of two problematic items:
    1. "${PN}" - instead "${BPN}" should be used to avoid problems when PN is prefixed or suffixed e.g. with multiconfig.
    2. github archive tarballs - these are not guaranteed to be stable and thus it is likely that they will be refreshed and thus the SRC_URI checksums will fail to apply. It is recommended that you fetch either an official release tarball or a specific revision from the actual git repository instead.
    Either one of these will now trigger a warning by default. If you wish to disable this check, remove src-uri-bad from WARN_QA.
  • The file-rdeps runtime dependency check no longer expands RDEPENDS recursively as there is no mechanism to ensure they can be fully computed, and thus races sometimes resulted in errors showing up and sometimes not. Thus you may now see errors for missing runtime dependencies that were previously satisfied recursively e.g. package A contains a shell script starting with #!/bin/bash but has no dependency on bash, but depends on package B that does depend on bash. Add the missing dependency / dependencies to resolve the warning.
  • Setting DEPENDS_${PN} anywhere (typically in a recipe) will now trigger an error - this is invalid, as DEPENDS is not a package-specific variable (unlike RDEPENDS). Set DEPENDS instead.
  • systemd is currently known not to work well with the musl C library as upstream only officially supports linking it with glibc. Thus, a warning will now be shown when building systemd in conjunction with musl.

Miscellaneous

  • The gnome class has been removed, as it did not do very much anymore. Recipes that previously inherited this class should be updated to do the following:
inherit gnomebase gtk-icon-cache gconf mime
  • The meta/recipes-kernel/linux/linux-dtb.inc file has been removed. It was previously deprecated (in favour of setting KERNEL_DEVICETREE in any kernel recipe) and only produced a warning. Remove any include/require statements pointing to this file.
  • TARGET_CFLAGS, TARGET_CPPFLAGS, TARGET_CXXFLAGS, and TARGET_LDFLAGS are no longer exported to the external environment. This has not required changes to any recipes, but if for some reason the software being built by one of your recipes is expecting these variables to be set then it will now fail, and you will need to either export the variable(s) in the recipe or change the scripts so that that isn't necessary.
  • The host distro identifier as used in NATIVELSBSTRING (when not using uninative) and SANITY_TESTED_DISTROS is now always lowercased even if it does not contain a version number.
  • In the base-files recipe, writing the hostname into /etc/hosts and /etc/hostname is now done within the main do_install function rather than do_install_basefilesissue, so that do_install_basefilesissue can be more easily overridden without having to duplicate the hostname functionality. If you have done the latter (e.g. in a base-files bbappend) then you should remove it from your customised do_install_basefilesissue function.
  • The wic --expand command-line option now uses commas to separate key:value pairs rather than hyphens, as was documented in the command-line help. Any scripts or commands where you use wic --expand with multiple key:value pairs will need to be updated.
  • UEFI image variable settings have been moved from various places to a central conf/image-uefi.conf. However, this should not influence any existing configuration as the meta/conf/image-uefi.conf in the core metadata sets defaults that can be overridden in the same manner as before.
  • conf/distro/include/world-broken.inc has been removed - where we need to disable certain recipes when using the musl C library, individual recipes now have COMPATIBLE_HOST_libc-musl set with a comment explaining why.