Yocto Project - User contributions [en]

Extensible SDK

2017-11-20T23:15:10Z

Henry Bruce: /* Releases */

'''This page is being continually updated with more content. Please bookmark and come back soon.''' Any questions, contact [mailto:henry.bruce@intel.com Henry Bruce]

== Introduction ==
The Yocto Project Extensible SDK (eSDK) has tools that allow you to easily add new applications and libraries to an image, modify the source of an existing component and test changes on the target hardware. The main benefit over the standard SDK is improved team workflow due to tighter integration with the OpenEmbedded build system.

Want to learn more about the Extensible SDK? You've come to the right place.

== Documentation ==
To learn about using the Extensible SDK read the [http://www.yoctoproject.org/docs/current/sdk-manual/sdk-manual.html SDK manual]. Information on building and customizing an eSDK can be found in [http://www.yoctoproject.org/docs/current/sdk-manual/sdk-manual.html#sdk-appendix-customizing Appendix B]. Also see this [http://events.linuxfoundation.org/sites/events/files/slides/2017%20ELC%20Henry%20Bruce.pdf eSDK talk] and companion [https://events.linuxfoundation.org/sites/events/files/slides/2017%20ELC%20--%20Using%20devtool%20to%20Streamline%20your%20Yocto%20Project%20Workflow.pdf devtool talk].

== Pre-built Installers ==
The Yocto Project auto-builder creates core-image-minimal and core-image-0sato eSDK installers for machines defined in oe-core. Currently only full installers are built. See http://autobuilder.yoctoproject.org/pub/releases/yocto-2.2.1/toolchain/x86_64. As an example [http://autobuilder.yoctoproject.org/pub/releases/yocto-2.2.1/toolchain/x86_64/poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-2.2.1.sh poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-2.2.1.sh] is the core-image-sato eSDK installer for machine genericx86-64.

== Tips and Tricks ==
* [[TipsAndTricks/Cmake,Eclipse,_and_SDKS | Add cmake to your eSDK]]

== Workflow ==
Details of the improved Yocto Project workflow enabled by using containers and the eSDK can be found on the [[Developer Workflow Improvements]] page.

== Presentations ==
=== ELC 2017 ===
* [http://events.linuxfoundation.org/sites/events/files/slides/2017%20ELC%20Henry%20Bruce.pdf Henry Bruce's eSDK talk]
* [https://events.linuxfoundation.org/sites/events/files/slides/2017%20ELC%20--%20Using%20devtool%20to%20Streamline%20your%20Yocto%20Project%20Workflow.pdf Tim Orling's devtool talk]
* [https://events.linuxfoundation.org/sites/events/files/slides/2017%20ELC%20-%20Yocto%20Project%20Containers.pdf Randy Witt's container talk]

== Releases ==
* First appeared as part of fido (1.8).
* Jethro (2.0) is beta release.
* Krogoth (2.1) is first usable release.
* Morty (2.2) brings "devtool finish" that generates patches to go with recipe
* Pyro (2.3) focuses on bug fixes
* Rocko (2.4) brings devtool (and underlying recipetool) enhancements

== Community ==
The eSDK is part of [http://www.openembedded.org/wiki/OpenEmbedded-Core Openembedded Core].
* Mailing list: openembedded-core@lists.openembedded.org
* Bugs: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=extensible%20sdk&list_id=592532
* Yocto Project SDK QuickStart: https://mender.io/resources/yocto-sdk-quickstart

TipsAndTricks/EnablingAPackageFeed

2017-10-05T23:57:53Z

Henry Bruce: /* Pyro (2.3) and greater */

= Morty (2.2) and earlier =
== Caveat ==
This article is based on work done a few years ago with fido (1.8) and only covers using smart with rpm. Needs some testing and extended to cover using dnf.

Instructions assume you have had at least one successful build of your target image.

== Select Your Package Format ==
package format is set via variable PACKAGE_CLASSES, typically in local.conf. Look for line like
PACKAGE_CLASSES ?= "package_rpm"
This means you're using rpm.

== Know Your Package Architectures ==
Your OS image will be comprised of a number of different package architectures.
=== From the build system view ===
The package feed needs to know what they are so look in tmp/deploy/rpm
$ ls tmp/deploy/rpm/
all core2_32 edison x86_64_nativesdk
You can exclude anything starting with x86_64. This means the architectures on the target are: <tt>all core2_32 edison</tt>
=== From target system view ===
$ rpm -qq
look at the endings of the various packages. For example , if you build for qemux86-64 the suffixes will be
core2_64
qemux86_64
all
so those are the architectures supported on the target.

== Select Your Package Feed URL ==
Is this example we'll use <tt>http://my-server.com/repo</tt>

== Add Package Manager to Build ==
Smart package manager is in oe-core, so there is no need to add any extra layers. You do, however need to tell the build system to include pkg management on the target, since excluding package management saves space, it is not enabled by default.

EXTRA_IMAGE_FEATURES += " package-management "

On releases prior to jethro, you *may* need to also do
IMAGE_INSTALL_append=" python-smartrpm "

== Configure Package Feed in Build ==
Now you have the information to create a package feed, ideally you add details in your distro conf file. If you are not using your own distro, you can set the following in local.conf. Note, if you only want to configure your channels on the target and not have any defaults, these definitions can be excluded.
PACKAGE_FEED_URIS = "http://my-server.com/repo"
PACKAGE_FEED_BASE_PATHS = "rpm"
PACKAGE_FEED_ARCHS = "all edison core2_32"

== Create Package Feed ==
* Re-build image so it includes smart and the package feed info
** Also, if you find you need an additional recipe not included in the image e.g. gawk, you can do bitbake gawk and it's packages will be available too.
*** this can be done iteratively
* Update repo package indices
bitbake package-index
* Copy packages to server. This sample script assumes files are served from /var/www/html/repo
** We exclude x86_64 as those are host side (aka native) rpms
<pre>
rsync -r -u --exclude 'x86_64*' tmp/deploy/rpm/* /var/www/html/repo
</pre>
* If you don't have a web server already serving up a directory and don't feel like going through the effort to do that you can do the following.
** Assume your feed is /opt/feed
** apt-get/dnf install python-twistd
** run twistd on the directory
$ twistd -n web --path /opt/feed -p 5678
* This way you can serve different feeds on different ports

== Run Package Manager on Target ==
* Make sure network is up and sync with package feed.
smart update
* Install my-package
smart search my-package
smart install my-package

== Recover Package Feed Config On Target ==
Smart does not have a config file, so config must be done via command line. This script re-creates the config set by distro in example above
<pre>
feed_url="http://my-server.com/repo"
platform="my-platform"
repo_name="my-repo"
for arch in all edison core2_32; do
smart channel --add $platform-$arch type=rpm-md name="$repo_name" baseurl=$feed_url/$arch -y
done
</pre>
Rebuild the cache with the new channels added
$ smart update
This will result in a set of feeds you can see with
$ smart channel --show

= Pyro (2.3) and greater =
== From a Build dir ==
=== Easiest Way ===
* Use RPMS
** PACKAGE_CLASSES = "package_rpm" in local.conf
* have package tools
** EXTRA_IMAGE_FEATURES += " package-management " in local.conf
* build something you want
** bitbake mdadm
* update the package indices
** bitbake package-index
* Serve up feed
** twistd -n web --path tmp/deploy/rpm -p 5678
=== How to keep a separate feed Dir ===
* Use RPMS
** PACKAGE_CLASSES = "package_rpm" in local.conf
* have package tools
** EXTRA_IMAGE_FEATURES += " package-management " in local.conf
* build something you want
** bitbake mdadm
* copy to dir
** rsync -r -u --exclude 'x86_64*' tmp/deploy/rpm/* feedDir/
* Build the tools needed to make directories into repos
** bitbake createrepo-c-native -caddto_recipe_sysroot
* Turn the directory into a repo
** oe-run-native createrepo-c-native createrepo_c feedDir
* Serve up feed
** twistd -n web --path feedDir -p 5678

== From the Target ==
* make the target aware of the repo you just created.
** make a dir for repos
*** <pre> mkdir -p /etc/yum.repos.d </pre>
** add the repo. The file name must end in '.repo' for instance mybuild.repo
<pre>
[myrepo]
name=myRepo for testing
baseurl=http://<host ip addr or hostname>:5678
enabled=1
metadata_expire=0
gpgcheck=0
</pre>
* install something
<pre>
dnf -v install mdadm
</pre>

== Why Might I Use This? ==
This workflow is one where you need to iteratively try out what packages you want on your image. In my case, I was doing a native build of a complex sw repo and needed to make;fail;add dependency somehow;make again; quite a few times. I certainly did not want to wait while bitbake made a new image with my additional dependency and then wait again while dd copied that image to my usb key. This way, I could bitbake dependency, rerun createrepo, dnf install from target. Much much faster.

TipsAndTricks/EnablingAPackageFeed

2017-10-05T23:57:37Z

Henry Bruce: /* Create Package Feed */

= Morty (2.2) and earlier =
== Caveat ==
This article is based on work done a few years ago with fido (1.8) and only covers using smart with rpm. Needs some testing and extended to cover using dnf.

Instructions assume you have had at least one successful build of your target image.

== Select Your Package Format ==
package format is set via variable PACKAGE_CLASSES, typically in local.conf. Look for line like
PACKAGE_CLASSES ?= "package_rpm"
This means you're using rpm.

== Know Your Package Architectures ==
Your OS image will be comprised of a number of different package architectures.
=== From the build system view ===
The package feed needs to know what they are so look in tmp/deploy/rpm
$ ls tmp/deploy/rpm/
all core2_32 edison x86_64_nativesdk
You can exclude anything starting with x86_64. This means the architectures on the target are: <tt>all core2_32 edison</tt>
=== From target system view ===
$ rpm -qq
look at the endings of the various packages. For example , if you build for qemux86-64 the suffixes will be
core2_64
qemux86_64
all
so those are the architectures supported on the target.

== Select Your Package Feed URL ==
Is this example we'll use <tt>http://my-server.com/repo</tt>

== Add Package Manager to Build ==
Smart package manager is in oe-core, so there is no need to add any extra layers. You do, however need to tell the build system to include pkg management on the target, since excluding package management saves space, it is not enabled by default.

EXTRA_IMAGE_FEATURES += " package-management "

On releases prior to jethro, you *may* need to also do
IMAGE_INSTALL_append=" python-smartrpm "

== Configure Package Feed in Build ==
Now you have the information to create a package feed, ideally you add details in your distro conf file. If you are not using your own distro, you can set the following in local.conf. Note, if you only want to configure your channels on the target and not have any defaults, these definitions can be excluded.
PACKAGE_FEED_URIS = "http://my-server.com/repo"
PACKAGE_FEED_BASE_PATHS = "rpm"
PACKAGE_FEED_ARCHS = "all edison core2_32"

== Create Package Feed ==
* Re-build image so it includes smart and the package feed info
** Also, if you find you need an additional recipe not included in the image e.g. gawk, you can do bitbake gawk and it's packages will be available too.
*** this can be done iteratively
* Update repo package indices
bitbake package-index
* Copy packages to server. This sample script assumes files are served from /var/www/html/repo
** We exclude x86_64 as those are host side (aka native) rpms
<pre>
rsync -r -u --exclude 'x86_64*' tmp/deploy/rpm/* /var/www/html/repo
</pre>
* If you don't have a web server already serving up a directory and don't feel like going through the effort to do that you can do the following.
** Assume your feed is /opt/feed
** apt-get/dnf install python-twistd
** run twistd on the directory
$ twistd -n web --path /opt/feed -p 5678
* This way you can serve different feeds on different ports

== Run Package Manager on Target ==
* Make sure network is up and sync with package feed.
smart update
* Install my-package
smart search my-package
smart install my-package

== Recover Package Feed Config On Target ==
Smart does not have a config file, so config must be done via command line. This script re-creates the config set by distro in example above
<pre>
feed_url="http://my-server.com/repo"
platform="my-platform"
repo_name="my-repo"
for arch in all edison core2_32; do
smart channel --add $platform-$arch type=rpm-md name="$repo_name" baseurl=$feed_url/$arch -y
done
</pre>
Rebuild the cache with the new channels added
$ smart update
This will result in a set of feeds you can see with
$ smart channel --show

= Pyro (2.3) and greater =
== Quick and Dirty Feed Serving ==
If you don't have a web server already serving up a directory and don't feel like going through the effort to do that you can do the following.
* Assume your feed is /opt/feed
* apt-get/dnf install python-twistd
* run twistd on the directory
$ twistd -n web --path /opt/feed -p 5678
* This way you can serve diff feeds on diff ports easily

== From a Build dir ==
=== Easiest Way ===
* Use RPMS
** PACKAGE_CLASSES = "package_rpm" in local.conf
* have package tools
** EXTRA_IMAGE_FEATURES += " package-management " in local.conf
* build something you want
** bitbake mdadm
* update the package indices
** bitbake package-index
* Serve up feed
** twistd -n web --path tmp/deploy/rpm -p 5678
=== How to keep a separate feed Dir ===
* Use RPMS
** PACKAGE_CLASSES = "package_rpm" in local.conf
* have package tools
** EXTRA_IMAGE_FEATURES += " package-management " in local.conf
* build something you want
** bitbake mdadm
* copy to dir
** rsync -r -u --exclude 'x86_64*' tmp/deploy/rpm/* feedDir/
* Build the tools needed to make directories into repos
** bitbake createrepo-c-native -caddto_recipe_sysroot
* Turn the directory into a repo
** oe-run-native createrepo-c-native createrepo_c feedDir
* Serve up feed
** twistd -n web --path feedDir -p 5678

== From the Target ==
* make the target aware of the repo you just created.
** make a dir for repos
*** <pre> mkdir -p /etc/yum.repos.d </pre>
** add the repo. The file name must end in '.repo' for instance mybuild.repo
<pre>
[myrepo]
name=myRepo for testing
baseurl=http://<host ip addr or hostname>:5678
enabled=1
metadata_expire=0
gpgcheck=0
</pre>
* install something
<pre>
dnf -v install mdadm
</pre>

== Why Might I Use This? ==
This workflow is one where you need to iteratively try out what packages you want on your image. In my case, I was doing a native build of a complex sw repo and needed to make;fail;add dependency somehow;make again; quite a few times. I certainly did not want to wait while bitbake made a new image with my additional dependency and then wait again while dd copied that image to my usb key. This way, I could bitbake dependency, rerun createrepo, dnf install from target. Much much faster.

TipsAndTricks/EnablingAPackageFeed

2017-10-05T23:51:22Z

Henry Bruce:

= Morty (2.2) and earlier =
== Caveat ==
This article is based on work done a few years ago with fido (1.8) and only covers using smart with rpm. Needs some testing and extended to cover using dnf.

Instructions assume you have had at least one successful build of your target image.

== Select Your Package Format ==
package format is set via variable PACKAGE_CLASSES, typically in local.conf. Look for line like
PACKAGE_CLASSES ?= "package_rpm"
This means you're using rpm.

== Know Your Package Architectures ==
Your OS image will be comprised of a number of different package architectures.
=== From the build system view ===
The package feed needs to know what they are so look in tmp/deploy/rpm
$ ls tmp/deploy/rpm/
all core2_32 edison x86_64_nativesdk
You can exclude anything starting with x86_64. This means the architectures on the target are: <tt>all core2_32 edison</tt>
=== From target system view ===
$ rpm -qq
look at the endings of the various packages. For example , if you build for qemux86-64 the suffixes will be
core2_64
qemux86_64
all
so those are the architectures supported on the target.

== Select Your Package Feed URL ==
Is this example we'll use <tt>http://my-server.com/repo</tt>

== Add Package Manager to Build ==
Smart package manager is in oe-core, so there is no need to add any extra layers. You do, however need to tell the build system to include pkg management on the target, since excluding package management saves space, it is not enabled by default.

EXTRA_IMAGE_FEATURES += " package-management "

On releases prior to jethro, you *may* need to also do
IMAGE_INSTALL_append=" python-smartrpm "

== Configure Package Feed in Build ==
Now you have the information to create a package feed, ideally you add details in your distro conf file. If you are not using your own distro, you can set the following in local.conf. Note, if you only want to configure your channels on the target and not have any defaults, these definitions can be excluded.
PACKAGE_FEED_URIS = "http://my-server.com/repo"
PACKAGE_FEED_BASE_PATHS = "rpm"
PACKAGE_FEED_ARCHS = "all edison core2_32"

== Create Package Feed ==
* Re-build image so it includes smart and the package feed info
** Also, if you find you need an additional recipe not included in the image e.g. gawk, you can do bitbake gawk and it's packages will be available too.
*** this can be done iteratively
* Update repo package indices
bitbake package-index
* Copy packages to server. This sample script assumes files are served from /var/www/html/repo
** We exclude x86_64 as those are host side (aka native) rpms
<pre>
rsync -r -u --exclude 'x86_64*' tmp/deploy/rpm/* /var/www/html/repo
</pre>

== Run Package Manager on Target ==
* Make sure network is up and sync with package feed.
smart update
* Install my-package
smart search my-package
smart install my-package

== Recover Package Feed Config On Target ==
Smart does not have a config file, so config must be done via command line. This script re-creates the config set by distro in example above
<pre>
feed_url="http://my-server.com/repo"
platform="my-platform"
repo_name="my-repo"
for arch in all edison core2_32; do
smart channel --add $platform-$arch type=rpm-md name="$repo_name" baseurl=$feed_url/$arch -y
done
</pre>
Rebuild the cache with the new channels added
$ smart update
This will result in a set of feeds you can see with
$ smart channel --show

= Pyro (2.3) and greater =
== Quick and Dirty Feed Serving ==
If you don't have a web server already serving up a directory and don't feel like going through the effort to do that you can do the following.
* Assume your feed is /opt/feed
* apt-get/dnf install python-twistd
* run twistd on the directory
$ twistd -n web --path /opt/feed -p 5678
* This way you can serve diff feeds on diff ports easily

== From a Build dir ==
=== Easiest Way ===
* Use RPMS
** PACKAGE_CLASSES = "package_rpm" in local.conf
* have package tools
** EXTRA_IMAGE_FEATURES += " package-management " in local.conf
* build something you want
** bitbake mdadm
* update the package indices
** bitbake package-index
* Serve up feed
** twistd -n web --path tmp/deploy/rpm -p 5678
=== How to keep a separate feed Dir ===
* Use RPMS
** PACKAGE_CLASSES = "package_rpm" in local.conf
* have package tools
** EXTRA_IMAGE_FEATURES += " package-management " in local.conf
* build something you want
** bitbake mdadm
* copy to dir
** rsync -r -u --exclude 'x86_64*' tmp/deploy/rpm/* feedDir/
* Build the tools needed to make directories into repos
** bitbake createrepo-c-native -caddto_recipe_sysroot
* Turn the directory into a repo
** oe-run-native createrepo-c-native createrepo_c feedDir
* Serve up feed
** twistd -n web --path feedDir -p 5678

== From the Target ==
* make the target aware of the repo you just created.
** make a dir for repos
*** <pre> mkdir -p /etc/yum.repos.d </pre>
** add the repo. The file name must end in '.repo' for instance mybuild.repo
<pre>
[myrepo]
name=myRepo for testing
baseurl=http://<host ip addr or hostname>:5678
enabled=1
metadata_expire=0
gpgcheck=0
</pre>
* install something
<pre>
dnf -v install mdadm
</pre>

== Why Might I Use This? ==
This workflow is one where you need to iteratively try out what packages you want on your image. In my case, I was doing a native build of a complex sw repo and needed to make;fail;add dependency somehow;make again; quite a few times. I certainly did not want to wait while bitbake made a new image with my additional dependency and then wait again while dd copied that image to my usb key. This way, I could bitbake dependency, rerun createrepo, dnf install from target. Much much faster.

Documentation Review

2017-09-09T04:21:44Z

Henry Bruce: Created page with "Yocto documentation review."

Yocto documentation review.

TipsAndTricks/KernelDevelopmentWithEsdk

2017-08-01T17:48:59Z

Henry Bruce: /* Checkout kernel source */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives you an easy way of getting the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your '''bitbake terminal'''. First we'll checkout poky and configure bitbake environment.
<pre>
$ git clone -b pyro git://git.yoctoproject.org/poky
$ source poky/oe-init-build-env
</pre>
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
We also need to create a layer to take the kernel patches we'll be creating. Be sure to create example recipe and append files when you run you run the "yocto-layer" command.
<pre>
$ yocto-layer create my-kernel -o ../meta-my-kernel
Please enter the layer priority you'd like to use for the layer: [default: 6]
Would you like to have an example recipe created? (y/n) [default: n] y
Please enter the name you'd like to use for your example recipe: [default: example]
Would you like to have an example bbappend file created? (y/n) [default: n] y
Please enter the name you'd like to use for your bbappend file: [default: example]
Please enter the version number you'd like to use for your bbappend file (this should match the recipe you're appending to): [default: 0.1]

New layer created in ../meta-my-kernel.

Don't forget to add it to your BBLAYERS (for details see ../meta-my-kernel/README).
</pre>
As directed, we need to add our layer to BBLAYERS as follows:
<pre>
$ bitbake-layers add-layer ../meta-my-kernel
</pre>
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
<pre>
$ bitbake core-image-minimal -c populate_sdk_ext
</pre>
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.1sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that includes the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.1sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. If you do not want to use the default ~/poky_sdk directory for installation of the toolchain, use the -d option to specify a destination.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.1.sh
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3.1
============================================================================
Enter target directory for SDK (default: ~/poky_sdk):
You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed[Y/n]? Y
Extracting SDK......................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |#########################################################################################################| Time: 0:00:52
Initialising tasks: 100% |######################################################################################################| Time: 0:00:04
Checking sstate mirror object availability: 100% |##############################################################################| Time: 0:00:00
Parsing recipes: 100% |#########################################################################################################| Time: 0:00:33
Initialising tasks: 100% |######################################################################################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux
</pre>
Take note of the final line, which shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a
new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot the Minnowboard with this image and check to be sure that everything is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.

<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

There is a bug that may cause error messages such as the following to be displayed. These can be ignored, the source code will be correct checked out.
ERROR: Taskhash mismatch 2c793438c2d9f8c3681fd5f7bc819efa versus be3a89ce7c47178880ba7bf6293d7404 for /path/to/esdk/layers/poky/meta/recipes-kernel/linux/linux-yocto_4.10.bb.do_unpack

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

=== Export patches and create a bbappend file ===
To export your commits as patches and a bbappend file use the following command in the ESDK terminal. We'll target the "my-kernel" layer we created in the bitbake terminal
$ devtool finish linux-yocto /path/to/meta-my-kernel
The patches and the bbappend can be found in /path/to/meta-my-kernel/recipes-kernel/linux.

== Working with upstream kernel ==
Slightly different workflow where we use a kernel from kernel.org.

Content TBD

== Build image with your modified kernel ==
You can now build an image which will include your kernel patches.
Execute the following command from your build directory in the bitbake terminal
bitbake core-image-minimal

TipsAndTricks

2017-07-31T20:45:00Z

Henry Bruce: /* Articles in development */

== Background ==
This wiki page captures ideas for "Tips and tricks" articles that are aimed Yocto Project users that have a basic understanding of the core tools and want to extend their knowledge. Articles will written and published using the following process
* Articles must refer to the current (or earlier) release. They must not cover features in development in the master branch.
* Anyone can add entries to the '''Ideas for Articles''' section based on challenges they have encountered when using the Yocto Project
* More experienced developers can start to flesh out articles in the ''Ideas'' section move then to '''Articles in Development''' or contribute directly to this section.
* Approximately once a month an article ready for publishing will be chosen for publication by [mailto:jeffrey.osier-mixon@intel.com Jefro] and moved to the '''Finished Articles ''' section.

Please contact [mailto:henry.bruce@intel.com?Subject=Yocto%20Project%20Tips%20and%20Tricks Henry Bruce] with any questions.

== Ideas for Articles ==
* [[TipsAndTricks/GeneratingASDK]] (Ross, Paul help please...)
* [[TipsAndTricks/AddingALicense]] (Ross)
* [[TipsAndTricks/UsingBuildstatsDiff]] (Ross)
* [[TipsAndTricks/DevPyShell]] (Richard Purdie)
* [[TipsAndTricks/CreatingABSP]]. And what not to put in it.
* [[TipsAndTricks/Patchwork]]
* [[TipsAndTricks/Patchtest]] (Leo)
* [[TipsAndTricks/CreateNewLayer]] (Brendan)

== Articles in development ==
* [[TipsAndTricks/DockerOnImage]] (bavery)
* [[TipsAndTricks/OnTargetWorkFlowLeveragingRPMPackagefeeds]] (bavery)
* [[TipsAndTricks/BuildingAndRunningClearContainersonTarget]] (bavery)
* [[TipsAndTricks/KernelDevelopmentWithEsdk]] (Todor Minchev, Henry Bruce)
* [[TipsAndTricks/DebugNativeRecipeWithGdb]] (Joshua Lock)
* [[TipsAndTricks/Netconsole]] (Ross Burton)
* [[TipsAndTricks/ParsingProfiling]] (Richard Purdie)
* [[TipsAndTricks/DemystifyingTheLinuxYoctoKernel]] (Tom Zanussi)
* [[TipsAndTricks/Patching the source for a recipe]] (Paul Eggleton)
* [[TipsAndTricks/Incorporating closed source components]] (Paul Eggleton)
* [[TipsAndTricks/DebuggingAvoidingRebuilds]] (Richard Purdie)
* [[TipsAndTricks/GitBisectABitbake]] (Ross)
* [[TipsAndTricks/RunningEclipseAgainstBuiltImage]] (bavery)
* [[TipsAndTricks/PrelinkSomePointersAndWorkarounds]] (bavery)
* [[TipsAndTricks/RunningQemuOnMacOSX]] (Stephano)
* [[TipsAndTricks/Understanding what changed (diffsigs etc)]] (Joshua)
* [[TipsAndTricks/CropsCLIContainers]] (bavery)
* [[TipsAndTricks/QuickAndDirtyKernelConfig]] (bavery)
* [[TipsAndTricks/TestingToasterWithContainers]] (bavery)
* [[TipsAndTricks/InvestigatingBuildTime]] (Leo Sandoval)
* [[TipsAndTricks/ResolvingLocaleIssues]] (bavery)
* [[TipsAndTricks/DebuggingBitbakeInPudb]] (bavery)
* [[TipsAndTricks/DebuggingBitbakeInWingIDE]] (bavery)
* [[TipsAndTricks/CliBuildsInToasterBuilddir]] (bavery)
* [[TipsAndTricks/BuildingZephyrImages]] (Henry)
* [[TipsAndTricks/Cmake,Eclipse, and SDKS]] (bavery)
* [[TipsAndTricks/LinuxKernelAndSDKs]] (bavery)
* [[TipsAndTricks/Running YP binaries on Ubuntu and Vice Versa]] (bavery)
* [[TipsAndTricks/LockSharedState]] (Stephano)
* [[TipsAndTricks/Building and booting for Joule or MinnowBoard]] (Cal)
* [[TipsAndTricks/EnablingAPackageFeed]] (Henry)
* [[TipsAndTricks/Creating Recipes for ROS modules]] (Henry)
* [[TipsAndTricks/TeamWorkflows]] (Joshua)

== Finished Articles ==
* [[TipsAndTricks/Packaging Prebuilt Libraries | Packaging Prebuilt Libraries]] (Ross and Henry)
* [[TipsAndTricks/NPM | Packaging Node.js Projects]] (Brendan and Henry)

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:40:34Z

Henry Bruce: /* Build image with your modified kernel */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your '''bitbake terminal'''. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
We also need to create a layer to take the kernel patches we'll be creating.
$ yocto-layer create my-kernel -o ../meta-my-kernel
$ bitbake-layers add-layer ../meta-my-kernel
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

=== Export patches and create a bbappend file ===
To export your commits as patches and a bbappend file use the following command in the ESDK terminal. We'll target the "my-kernel" layer we created in the bitbake terminal
$ devtool finish linux-yocto /path/to/meta-my-kernel
The patches and the bbappend can be found in /path/to/meta-my-kernel/recipes-kernel/linux.

== Working with upstream kernel ==
Slightly different workflow where we use a kernel from kernel.org.

Content TBD

== Build image with your modified kernel ==
You can now build an image which will include your kernel patches.
Execute the following command from your build directory in the bitbake terminal
bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:39:34Z

Henry Bruce: /* Export patches and create a bbappend file */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your '''bitbake terminal'''. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
We also need to create a layer to take the kernel patches we'll be creating.
$ yocto-layer create my-kernel -o ../meta-my-kernel
$ bitbake-layers add-layer ../meta-my-kernel
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

=== Export patches and create a bbappend file ===
To export your commits as patches and a bbappend file use the following command in the ESDK terminal. We'll target the "my-kernel" layer we created in the bitbake terminal
$ devtool finish linux-yocto /path/to/meta-my-kernel
The patches and the bbappend can be found in /path/to/meta-my-kernel/recipes-kernel/linux.

== Build image with your modified kernel ==
You can now build an image which will include your kernel patches.
Execute the following command from your build directory in the bitbake terminal
bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:39:00Z

Henry Bruce: /* Build image with your modified kernel (bitbake terminal) */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your '''bitbake terminal'''. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
We also need to create a layer to take the kernel patches we'll be creating.
$ yocto-layer create my-kernel -o ../meta-my-kernel
$ bitbake-layers add-layer ../meta-my-kernel
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

== Export patches and create a bbappend file ==
To export your commits as patches and a bbappend file use the following command in the ESDK terminal. We'll target the "my-kernel" layer we created in the bitbake terminal
$ devtool finish linux-yocto /path/to/meta-my-kernel
The patches and the bbappend can be found in /path/to/meta-my-kernel/recipes-kernel/linux.

== Build image with your modified kernel ==
You can now build an image which will include your kernel patches.
Execute the following command from your build directory in the bitbake terminal
bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:38:05Z

Henry Bruce: /* Export patches and create a bbappend file (bitbake terminal) */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your '''bitbake terminal'''. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
We also need to create a layer to take the kernel patches we'll be creating.
$ yocto-layer create my-kernel -o ../meta-my-kernel
$ bitbake-layers add-layer ../meta-my-kernel
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

== Export patches and create a bbappend file ==
To export your commits as patches and a bbappend file use the following command in the ESDK terminal. We'll target the "my-kernel" layer we created in the bitbake terminal
$ devtool finish linux-yocto /path/to/meta-my-kernel
The patches and the bbappend can be found in /path/to/meta-my-kernel/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:29:29Z

Henry Bruce: /* Build Extensible SDK */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your '''bitbake terminal'''. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
We also need to create a layer to take the kernel patches we'll be creating.
$ yocto-layer create my-kernel -o ../meta-my-kernel
$ bitbake-layers add-layer ../meta-my-kernel
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:24:22Z

Henry Bruce: /* Build Extensible SDK */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your '''bitbake terminal'''. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:18:28Z

Henry Bruce: /* Create image with new kernel */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First make a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
Then create loopback devices for each partition in wic file
$ sudo kpartx -v -a /tmp/core-image-minimal-genericx86-64.wic
add map loop3p1 (253:6): 0 47446 linear /dev/loop3 2048
add map loop3p2 (253:7): 0 119356 linear /dev/loop3 51200
add map loop3p3 (253:8): 0 90112 linear /dev/loop3 170556
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:09:27Z

Henry Bruce:

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First maske a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
$ sudo kpartx -a /tmp/core-image-minimal-genericx86-64.wic
This will create a loopback device, see /dev/mapper to find out which device.
$ ls /dev/mapper/loop*
/dev/mapper/loop3p1 /dev/mapper/loop3p2 /dev/mapper/loop3p3
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-29T00:08:22Z

Henry Bruce: /* Edit and build driver */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4
=== Create image with new kernel ===
Normally you'd make a new image with <tt>devtool build-image</tt> but this can take a minute or so and messes with the kernel source folder. A faster option is to use kpartx to splice the new kernel into the image you have already built. First maske a copy of your wic file in say /tmp
$ cp tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic /tmp
$ sudo kpartx -a /tmp/core-image-minimal-genericx86-64.wic
This will create a loopback device, see /dev/mapper to find out which device.
$ ls /dev/mapper/loop*
/dev/mapper/loop3p1 /dev/mapper/loop3p2 /dev/mapper/loop3p3
Kernel is in the first device, so let's mount /dev/mapper/loop3p1 to take a look
$ sudo mkdir /mnt/wic-p1
$ sudo mount /dev/mapper/loop3p1 /mnt/wic-p1
Now copy over new kernel
$ sudo cp workspace/sources/linux-yocto/arch/x86/boot/bzImage /mnt/wic-p1
Then unmount and unkaprtx...
$ sudo umount mnt/wic-p1
$ sudo kaprtx -d /dev/loop3
Now flash image
$ dd if=/tmp/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard, log on and look in log for driver message
# root@genericx86-64:~# dmesg | grep Serial:
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Serial: 8250/16550 driver modified with eSDK
It worked!

== Setup your ESDK build environment (ESDK terminal) ==

Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'

. /path_to_esdk/environment-setup-corei7-64-poky-linux

NOTE: If you see the message below you will really need to open a new terminal :).

"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."

== Build your kernel (ESDK terminal) ==

Navigate to your workspace directory e.g /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto
and build the kernel

make

== Modify your kernel and commit your changes (ESDK terminal)==

You can navigate to the Linux source directory (/path_to_build_directory/kernel-dev/workspace/sources/linux-yocto) and modify the kernel. Commit your kernel changes.

== Build an image with your kernel changes (optional) (bitbake terminal) ==

devtool build-image core-image-minimal

For faster iteration, in 2.4 some wic commands are being included to allow you to insert the kernel in an image without going through the full recreation of the image. For older releases you can do:
sudo kpartx -a <foo>.wic
sudo mount /dev/mapper/loopNpM /mnt/wic-partionM
In this case, it will be the next available loopback device. If you are not using any , it will be loop0. so in the std case it would look like
sudo mount /dev/mapper/loop0p1 /mnt/wic-partion1
Then you could dd the kernel into the kernel partion. Then unmount and unkaprtx...
sudo umount mnt/wic-partionM
sudo kaprtx -d /dev/loopN

Note that this requires root/sudo rights. The assumption here is most kernel devs would have this or could do it in a vm/container.

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T23:03:31Z

Henry Bruce: /* Build and test the kernel */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Edit and build driver ===
Kernel source will be in the location shown at end of the <tt>devtool modify virtual/kernel</tt> output. Let's make a trivial change to the serial driver in drivers/tty/serial/8250/8250_core.c. After the line
pr_info("Serial: 8250/16550 driver, %d ports, IRQ sharing %sabled\n",
nr_uarts, share_irqs ? "en" : "dis");
Add the line
pr_info("Serial: 8250/16550 driver modified with eSDK");
To build the updated kernel source, in the SDK terminal run make in your kernel source folder. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4

== Setup your ESDK build environment (ESDK terminal) ==

Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'

. /path_to_esdk/environment-setup-corei7-64-poky-linux

NOTE: If you see the message below you will really need to open a new terminal :).

"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."

== Build your kernel (ESDK terminal) ==

Navigate to your workspace directory e.g /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto
and build the kernel

make

== Modify your kernel and commit your changes (ESDK terminal)==

You can navigate to the Linux source directory (/path_to_build_directory/kernel-dev/workspace/sources/linux-yocto) and modify the kernel. Commit your kernel changes.

== Build an image with your kernel changes (optional) (bitbake terminal) ==

devtool build-image core-image-minimal

For faster iteration, in 2.4 some wic commands are being included to allow you to insert the kernel in an image without going through the full recreation of the image. For older releases you can do:
sudo kpartx -a <foo>.wic
sudo mount /dev/mapper/loopNpM /mnt/wic-partionM
In this case, it will be the next available loopback device. If you are not using any , it will be loop0. so in the std case it would look like
sudo mount /dev/mapper/loop0p1 /mnt/wic-partion1
Then you could dd the kernel into the kernel partion. Then unmount and unkaprtx...
sudo umount mnt/wic-partionM
sudo kaprtx -d /dev/loopN

Note that this requires root/sudo rights. The assumption here is most kernel devs would have this or could do it in a vm/container.

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T20:51:47Z

Henry Bruce: /* = Build and test the kernel */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Build and test the kernel ===
In the SDK terminal, run make in your kernel source folder as given at end of <tt>devtool modify</tt> output. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4

== Setup your ESDK build environment (ESDK terminal) ==

Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'

. /path_to_esdk/environment-setup-corei7-64-poky-linux

NOTE: If you see the message below you will really need to open a new terminal :).

"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."

== Build your kernel (ESDK terminal) ==

Navigate to your workspace directory e.g /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto
and build the kernel

make

== Modify your kernel and commit your changes (ESDK terminal)==

You can navigate to the Linux source directory (/path_to_build_directory/kernel-dev/workspace/sources/linux-yocto) and modify the kernel. Commit your kernel changes.

== Build an image with your kernel changes (optional) (bitbake terminal) ==

devtool build-image core-image-minimal

For faster iteration, in 2.4 some wic commands are being included to allow you to insert the kernel in an image without going through the full recreation of the image. For older releases you can do:
sudo kpartx -a <foo>.wic
sudo mount /dev/mapper/loopNpM /mnt/wic-partionM
In this case, it will be the next available loopback device. If you are not using any , it will be loop0. so in the std case it would look like
sudo mount /dev/mapper/loop0p1 /mnt/wic-partion1
Then you could dd the kernel into the kernel partion. Then unmount and unkaprtx...
sudo umount mnt/wic-partionM
sudo kaprtx -d /dev/loopN

Note that this requires root/sudo rights. The assumption here is most kernel devs would have this or could do it in a vm/container.

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T20:51:35Z

Henry Bruce: /* Prepare to make changes to kernel */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Working with Yocto kernel ==
=== Checkout kernel source ===
First we must use devtool to checkout the kernel source code in its workspace. Do the following in the ESDK terminal. Note that we use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Build and test the kernel ==
In the SDK terminal, run make in your kernel source folder as given at end of <tt>devtool modify</tt> output. Adjust the -j option to match number of cores on your workstation.
$ make -C workspace/sources/linux-yocto -j4

== Setup your ESDK build environment (ESDK terminal) ==

Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'

. /path_to_esdk/environment-setup-corei7-64-poky-linux

NOTE: If you see the message below you will really need to open a new terminal :).

"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."

== Build your kernel (ESDK terminal) ==

Navigate to your workspace directory e.g /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto
and build the kernel

make

== Modify your kernel and commit your changes (ESDK terminal)==

You can navigate to the Linux source directory (/path_to_build_directory/kernel-dev/workspace/sources/linux-yocto) and modify the kernel. Commit your kernel changes.

== Build an image with your kernel changes (optional) (bitbake terminal) ==

devtool build-image core-image-minimal

For faster iteration, in 2.4 some wic commands are being included to allow you to insert the kernel in an image without going through the full recreation of the image. For older releases you can do:
sudo kpartx -a <foo>.wic
sudo mount /dev/mapper/loopNpM /mnt/wic-partionM
In this case, it will be the next available loopback device. If you are not using any , it will be loop0. so in the std case it would look like
sudo mount /dev/mapper/loop0p1 /mnt/wic-partion1
Then you could dd the kernel into the kernel partion. Then unmount and unkaprtx...
sudo umount mnt/wic-partionM
sudo kaprtx -d /dev/loopN

Note that this requires root/sudo rights. The assumption here is most kernel devs would have this or could do it in a vm/container.

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T16:41:30Z

Henry Bruce: /* Prepare to make changes to kernel */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Prepare to make changes to kernel ==
First we must use devtool to checkout the kernel source code in its workspace. Note that use the virtual kernel provider so we don't need to know the kernel recipe name.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Checkout the kernel source tree and construct .config ===
Check out the Linux kernel source and construct a .config from the Yocto kernel configuration segments. The sources can be found in /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto.

devtool modify linux-yocto

If you are just using a kernel from somewhere else, for example kernel.org, you can just clone it instead of using devtool modify in this step.

== Setup your ESDK build environment (ESDK terminal) ==

Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'

. /path_to_esdk/environment-setup-corei7-64-poky-linux

NOTE: If you see the message below you will really need to open a new terminal :).

"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."

== Build your kernel (ESDK terminal) ==

Navigate to your workspace directory e.g /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto
and build the kernel

make

== Modify your kernel and commit your changes (ESDK terminal)==

You can navigate to the Linux source directory (/path_to_build_directory/kernel-dev/workspace/sources/linux-yocto) and modify the kernel. Commit your kernel changes.

== Build an image with your kernel changes (optional) (bitbake terminal) ==

devtool build-image core-image-minimal

For faster iteration, in 2.4 some wic commands are being included to allow you to insert the kernel in an image without going through the full recreation of the image. For older releases you can do:
sudo kpartx -a <foo>.wic
sudo mount /dev/mapper/loopNpM /mnt/wic-partionM
In this case, it will be the next available loopback device. If you are not using any , it will be loop0. so in the std case it would look like
sudo mount /dev/mapper/loop0p1 /mnt/wic-partion1
Then you could dd the kernel into the kernel partion. Then unmount and unkaprtx...
sudo umount mnt/wic-partionM
sudo kaprtx -d /dev/loopN

Note that this requires root/sudo rights. The assumption here is most kernel devs would have this or could do it in a vm/container.

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T16:29:53Z

Henry Bruce: /* Setup your ESDK build environment (ESDK terminal) */

== Overview ==
This article describes an efficient workflow for Linux kernel development using Yocto Project. The goal is to get the build, deploy and test cycle to under a minute. We will be working with the generix86-64 BSP and the Intel Minnowboard Turbot in this article but the process is the same for any platform and CPU architecture.

The extensible SDK (eSDK) is a key part of this workflow for a number of reasons
# It contains the target toolchain which is not easily accessible in the bitbake environment
# It contains '''devtool''' which gives use an easy way of getting the the kernel source your target is using (if using linux-yocto or linux-intel)
# devtool will also automatically create patches for your kernel updates and add them to the the kernel recipe

There are two separate steps to this process
# Build or download an eSDK for your platform
# Install and use that eSDK to build your kernel

== Build Extensible SDK ==
Open a new terminal window, we'll refer to this terminal as your 'bitbake terminal'. First we'll checkout poky and configure bitbake environment.
$ git clone -b pyro git://git.yocoproject.org/poky
$ source poky/oe-init-build-env
Now we need to do some configuration for the Minnowboard image.
* Set MACHINE (as default is qemux86)
* Enable kernel modules (disabled by default for minimal images)
Add the following to conf/local.conf
MACHINE = "genericx86-64"
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
Now build an extensible SDK installer for the Minnowboard in the bitbake terminal
$ bitbake core-image-minimal -c populate_sdk_ext
The eSDK installer can be found in /path_to_build_directory/kernel-dev/tmp/deploy/sdk/
./tmp/deploy/sdk/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh

Alternatively, if you are not using a yocto-linux kernel, you can use a prebuilt eSDK installer that included the toolchain you need. For example, the pyro core-image-minimal esdk release for core2-64 is located [[http://downloads.yoctoproject.org/releases/yocto/yocto-2.3/toolchain/x86_64/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.3.sh here]]. If you downloaded the installer, make it executable.

== Install Extensible SDK ==
Run the installer as follows. Note use of -d option to specify destination. By default installer wants to use /opt/poky that requires administrator privileges.
<pre>
$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.3.sh -d /path/to/esdk
Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.3
==========================================================================
You are about to install the SDK to "/home/hbruce/Tools/yocto/kernel". Proceed[Y/n]? Y
Extracting SDK...............................................done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100% |##########################################| Time: 0:00:08
Initialising tasks: 100% |#######################################| Time: 0:00:06
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
Parsing recipes: 100% |##########################################| Time: 0:00:05
Initialising tasks: 100% |#######################################| Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
$ . /home/hbruce/Tools/yocto/kernel/environment-setup-corei7-64-poky-linux
</pre>
Take note of the final line, this shows you the environment setup script you must run each time you want to use the eSDK.

== Setup your ESDK build environment (ESDK terminal) ==
'''YOU MUST OPEN A NEW TERMINAL WHEN USING THE ESDK. DO NOT RE-USE YOUR BITBAKE TERMINAL.'''
Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'. Run the setup script given by the installer.
$ source /path/to/esdk/environment-setup-core2-64-poky-linux
"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.

If you see this
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."
You didn't open a new terminal!

== Build initial image with ESDK ==
In the ESDK terminal, build the image
<pre>
$ devtool build-image
Parsing recipes: 100% |##########################################| Time: 0:00:05
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
WARNING: No packages to add, building image core-image-minimal unmodified
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#######################################| Time: 0:00:07
Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
NOTE: Successfully built core-image-minimal. You can find output files in /path/to/esdk/tmp/deploy/images/genericx86-64
</pre>
Now flash to image to USB stick, assuming USB stick is on /dev/sdd
$ sudo dd if=tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.wic of=/dev/sdd bs=1MB
$ sync
Boot Minnowboard with this image and check all is OK.

== Prepare to make changes to kernel ==
First we must use devtool to checkout the kernel source code in its workspace.
<pre>
$ devtool modify virtual/kernel
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Mapping virtual/kernel to linux-yocto
Loading cache: 100% |############################################| Time: 0:00:00
Loaded 1299 entries from dependency cache.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_fetch...
NOTE: Executing do_unpack...
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Executing RunQueue Tasks
NOTE: Executing do_kernel_checkout...
NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and all succeeded.
NOTE: Patching...
NOTE: Executing RunQueue Tasks
NOTE: Executing do_validate_branches...
NOTE: Executing do_kernel_metadata...
NOTE: Executing do_patch...
NOTE: Tasks Summary: Attempted 6 tasks of which 3 didn't need to be rerun and all succeeded.
NOTE: Generating kernel config
NOTE: Executing RunQueue Tasks
NOTE: Executing do_prepare_recipe_sysroot...
NOTE: Executing do_kernel_configme...
NOTE: Executing do_configure...
NOTE: Tasks Summary: Attempted 9 tasks of which 6 didn't need to be rerun and all succeeded.
NOTE: Copying kernel config to srctree
NOTE: Source tree extracted to /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
NOTE: Recipe linux-yocto now set up to build from /home/hbruce/Tools/yocto/kernel/workspace/sources/linux-yocto
</pre>

=== Checkout the kernel source tree and construct .config ===
Check out the Linux kernel source and construct a .config from the Yocto kernel configuration segments. The sources can be found in /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto.

devtool modify linux-yocto

If you are just using a kernel from somewhere else, for example kernel.org, you can just clone it instead of using devtool modify in this step.

== Setup your ESDK build environment (ESDK terminal) ==

Open a new terminal and setup your build environment. We'll refer to this terminal as your 'ESDK terminal'

. /path_to_esdk/environment-setup-corei7-64-poky-linux

NOTE: If you see the message below you will really need to open a new terminal :).

"SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead."

== Build your kernel (ESDK terminal) ==

Navigate to your workspace directory e.g /path_to_build_directory/kernel-dev/workspace/sources/linux-yocto
and build the kernel

make

== Modify your kernel and commit your changes (ESDK terminal)==

You can navigate to the Linux source directory (/path_to_build_directory/kernel-dev/workspace/sources/linux-yocto) and modify the kernel. Commit your kernel changes.

== Build an image with your kernel changes (optional) (bitbake terminal) ==

devtool build-image core-image-minimal

For faster iteration, in 2.4 some wic commands are being included to allow you to insert the kernel in an image without going through the full recreation of the image. For older releases you can do:
sudo kpartx -a <foo>.wic
sudo mount /dev/mapper/loopNpM /mnt/wic-partionM
In this case, it will be the next available loopback device. If you are not using any , it will be loop0. so in the std case it would look like
sudo mount /dev/mapper/loop0p1 /mnt/wic-partion1
Then you could dd the kernel into the kernel partion. Then unmount and unkaprtx...
sudo umount mnt/wic-partionM
sudo kaprtx -d /dev/loopN

Note that this requires root/sudo rights. The assumption here is most kernel devs would have this or could do it in a vm/container.

== Export patches and create a bbappend file (bitbake terminal) ==
To export your commits to patches and a bbappend file use the following command.

devtool finish linux-yocto /path_to_poky_directory/meta-yocto-bsp/

The patches and the bbappend can be found in /path_to_poky_directory/meta-yocto-bsp/recipes-kernel/linux.

== Build image with your modified kernel (bitbake terminal) ==

You can now build an image which will include your kernel patches.

Execute the following command from your build directory ( e.g /path_to_build_directory/kernel-dev/)

bitbake core-image-minimal

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T16:07:48Z

Henry Bruce: /* Setup your ESDK build environment (ESDK terminal) */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T16:07:04Z

Henry Bruce: /* Build Extensible SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-28T00:41:52Z

Henry Bruce: /* Build Extensible SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-27T22:03:10Z

Henry Bruce: /* Build Extensble SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-07-27T21:55:24Z

Henry Bruce: /* Overview */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T23:50:18Z

Henry Bruce: /* Install Extensible SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T23:49:07Z

Henry Bruce: /* Install Extensible SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T23:47:20Z

Henry Bruce: /* Checkout the kernel source tree and construct .config */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T23:20:52Z

Henry Bruce: /* Install Extensible SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T23:20:40Z

Henry Bruce: /* Build Extensble SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T22:57:18Z

Henry Bruce: /* Overview */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T22:54:13Z

Henry Bruce: /* Install Extensible SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T22:51:48Z

Henry Bruce: /* Build Extensble SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T22:41:43Z

Henry Bruce: /* Overview */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T22:41:05Z

Henry Bruce: /* Overview */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T22:40:28Z

Henry Bruce: /* Build Extensble SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T22:34:41Z

Henry Bruce: /* Build Extensble SDK */

TipsAndTricks/KernelDevelopmentWithEsdk

2017-06-27T20:36:15Z

Henry Bruce: /* Overview */

Glossary of Terms and Acronyms

2017-06-24T00:02:31Z

Henry Bruce: /* Terminology */

Back to Yocto Project [[Main Page]].

== Terminology ==
Here is a list of basic terms users new to the Yocto Project might find helpful. For a more complete list of terms, see the
"[http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#yocto-project-terms Yocto Project Terms]" section in the Yocto Project Development Manual.
:'''Yocto Project''': A Linux Foundation project that acts as an umbrella for various efforts to improve Embedded Linux.
: '''OpenEmbedded''': The build system architecture promoted by the Yocto Project.
: '''BitBake''': A tool that reads metadata and runs tasks.
: '''OpenEmbedded-Core''': The common base set of metadata that BitBake uses. This metadata is shared between OpenEmbedded and the Yocto Project.
: '''Poky''': A reference distribution used for test and release purposes by the Yocto Project.
: '''Recipe''': Meta data defining how bitbake is to configure, build and package a software project.
: '''Layer''': A collection of recipes and/or configuration used by bitbake. Typically each layer is organised around a specific theme, e.g. adding recipes for building web browser software.

Major Yocto Project versions release on a six-month cycle in spring and fall/autumn. Releases tend to known by their name rather than release number.
{| class="wikitable" border="2"
!Codename
!Version
!Release Date
|-
|Pyro
|2.3
|May 2017
|-
|Morty
|2.2
|Oct 2016
|-
|Krogoth
|2.1
|May 2016
|-
|Jethro
|2.0
|Nov 2015
|-
|Fido
|1.8
|Apr 2015
|-
|Dizzy
|1.7
|Oct 2014
|-
|Daisy
|1.6
|Apr 2014
|-
|}

== Acronyms ==
:'''BSP''' Board Support Package
: '''CCB''' Change Control Board
: '''PMP''' Program Management Plan
: '''POR''' Plan of Record
: '''PRD''' Product Requirements Document

Development Manual Rework

2017-06-23T22:29:34Z

Henry Bruce: /* Basic "How To" Topics */

= Intro =
This page will propose an outline for [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html Development Manual] refactoring work. See also [https://bugzilla.yoctoproject.org/show_bug.cgi?id=11630 bug #11630]

= Henry's Input =
I believe the manual lost its way as it became the dumping ground for useful (and often advanced) info that had nowhere else to land. As a result the size of the manual and top level topics became overwhelming to the new developer who had just graduated from the quick state guide. Currently this user is not served by the Yocto documentation, thus the creation of the wiki [[cookbook]]. Simple tasks such as adding an existing recipe to image and creating a recipe source should be the bread and butter of the development manual. If this approach makes the development master server too many master, we should dicuss re-structing all the docs the addition of a new one.

== Basic "How To" Topics ==
I plan to structure these, but just a plain list for the time being. Many of these topics are already documented, but this feels like a better flow that we have just now.
=== Basic Skills for Systems Developers ===
* Deal with corporate proxies.
* Add a package to an image
* Understand the difference between a recipe and package
* Build a package by itself and why that's useful
* Find out what packages are created by a recipe
* Find out what files are in a package
* Find out what files are in an image
* Add an ssh server to an image (enable transferring of files to target)
* Anatomy of a recipe
* How to find recipes (layers.openembedded,org)
* Understand difference between machine and distro settings
* How to find and use the right BSP (machine) for your hardware
* Examples of distro features and where to set them.
* Understanding the task pipeline and executing individual tasks
* Understand devtool and how it simplifies your workflow
* Improve build speeds with shared downloads and shared state cache
* Generate and understand dependency graph
* Generate and understand bitbake environment
* Building an Extensible SDK
=== Basic Skills for Application Developers ===
* Understand intent and content of extensible SDK (eSDK) and why traditional SDK is no longer necessary
* Understand CROPS eSDK container solution
** Source code is edited outside of container using tools of your choice on host
**
* Install and run CROPS/eSDK
* Intro to devtool
* Make sure you're up to date
* Build the image (may not be the same as on your target) and typical options for flashing it
* Modify an existing recipe
** Understand how git is managing source code
** Build and deploy modified recipe
** Using devtool finish to create updated recipe and patches.
** Understand how this is done without updating upstream source code
* Add a new recipe
** Create recipe from source code
** Review recipe to see what's missing
** Repeat steps as for 'modify'

= Tracey's Input =
The way Henry described this yesterday, customers use the "getting started" guide to learn how to bring up a vanilla image, but then don't know how to approach their own custom situation. This should be a TRANSITION DOCUMENT - how to use YP once you have gotten your feet wet in a controlled situation. Feel free to delete this, I don't know how you folks work in the WIKI and I'm assuming this may not be it.

I would suggest organizing this in a couple of ways - and this might be better in a google doc, frankly.
1) by what people need to do to get started
a) where to start to support a platform
1. find a BSP and test it
b) how to add support for platform components
c) how to slim down an image size
1. Remove code automatically included
2. design your own BSP fine tuned to exclude extra code, unneeded support code
d) How to approach setting up a development environment
e) How use layers to create a flexible long term design, and recipes
d) How do you start a new project vs build off an existing one
e) How do you set up a team sharable development environment (sstate)
f) Here are some of the most common commands
g) How to deal with CVEs and code branches

2) By helping them collect the information they need in order to make set up decisions
Brent had a great idea to make this a SW Wizard that would step a new user through a series of questions and give them at least one solution on how to proceed.
a) what platform are you using
b) will you include an application as part of the image or will it run on the image
c) what is the configuration of the team (system developers, app developers, etc)
c) How to help them choose the right kernel
d) etc.

= Scott's Input =

Scott Rifenbark - This is a good idea. I was thinking of a similar way to deal with this with sort of a decision tree at the end of the YP Quick Start (flowchart for old-schoolers) that would lead the reader down a path based on information they have about what they want to get accomplished. Then, based on how they traverse the tree, we point them to areas they need to use in the manual.

== Notes for Chapter 1 of the dev-manual: ==

I re-wrote this introductory section to work more tightly with the introductory material of the ref-manual. Also, created a better "Other Information" section to lead the reader to various places of interest. In particular, I am leveraging off the Yocto Project Backgrounders that exist on the YP Website. These backgrounders are probably the best place for an overall introduction to what the YP is supposed to be and do for you.

== Notes for Chapter 3 of the dev-manual: ==

This chapter was mostly reference material. The entire chapter has been converted now to procedures by pulling out the reference stuff and placing it into the ref-manual. What is left in Chapter 3 are procedures related to setting up a shared YP development environment, submitting a defect, and submitting a change.

=== Section 3.1 - "Open Source Philosophy" ===

I moved this section to the ref-manual. It now is in a chapter called "The Yocto Project Development Environment". That new chapter in the ref-manual is really the old "Closer Look" chapter now with some YP development environment concepts placed on top in their own subsections.

=== Section 3.2 - "Using the Yocto Project in a Team Environment" ===

I have converted this section to a procedure. I need more information on this section from the team. Emails are out soliciting input.

=== Section 3.3 - "Source Repositories" ===

This section has been moved to the ref-manual under "The Yocto Project Development Environment" section. No procedures here except perhaps "Locating the Yocto Project Source Repositories".

=== Section 3.4 - "Terms" ===

This section has been moved to the ref-manual into the "Introduction" chapter. All links throughout the YP Documentation Manual set have been adjusted to work for the new location.

=== Section 3.5 - "Licensing" ===

I moved this section to the ref-manual under "The Yocto Project Development Environment". There are no possibilities of procedures in this section. It is pure reference material.

=== Section 3.6 - "Git" ===

The concepts and overview of how Git works is best suited for the ref-manual. I moved the concepts of Git to the ref-manual. In doing so, I decided to keep the two small examples in that section there just for completeness. However, I created a new section in the dev-manual called "Working With Git Repositories" that has pointed procedures that show the reader how to clone poky, check out a branch based on a poky branch name, and check out a branch based on a poky tag. Right now in the "Getting Set Up" section but will likely be moved as the new dev-manual gains more procedures.

=== Section 3.7 - "Workflows" ===

I have moved this section to "The Yocto Project Development Environment" section in the ref-manual. The section is concepts only.

=== Section 3.8 - "Submitting a Defect Against the Yocto Project" ===

I have reworked this section and it is done. It will live in the dev-manual.

=== Section 3.9 - "How to Submit a Change" ===

I have re-worked this section into a couple of procedures.

== Notes for Chapter 4 of the dev-manual: ==

This whole section is about various workflows. The information is a mix of concepts, high-level workflow steps, and various details associated with the flows. I think the entire chapter can pretty much be recast in some conceptual area of the ref-manual. Then, for each type of workflow, we can create a specific task that has some meaningful steps and real examples for use in the dev-manual. These sections in the dev-manual can be referenced from the associated concepts in the ref-manual.

=== Section 4.1 - "System Development Workflow" ===

Introductory material that will live with the subsections for concepts (in the ref-manual) or introductory area for grouped tasks in the dev-manual.

=== Section 4.1.1 - "Developing a Board Support Package (BSP)" ===

Although the steps in this section are technically procedural, the level is more conceptual. We should move this discussion to the ref-manual and group it under some sort of new "concepts" heading where conceptual information is discussed. That is one idea. Another, is to create a conceptual section in the BSP Guide to hold this information. One thing we have not discussed is the fate of the BSP Guide in regards to the restructuring of the dev-manual to a task-based document. In any case, the current "Developing a Board Support Package (BSP)" section in the dev-manual is out as it stands now. If we want to replace it in the dev-manual with something more concrete, then I suggest a real example that would be in the form of what is on this dated wiki page - https://wiki.yoctoproject.org/wiki/Transcript:_creating_one_generic_Atom_BSP_from_another.

=== Section 4.1.2 - "Modifying the Kernel" ===

The information in this section is very conceptual and does not belong in the dev-manual. Solution for this section is similar to what I suggested for the "Developing a Board Support Package (BSP)" section above. However, this is kernel related so perhaps moving this kernel information to the kernel-dev manual. Again, if we want to have some concrete example in the dev-manual that is based on the conceptual information here, then we create a real example and use that and tie it to specific tasks.

=== Section 4.2 - "Application Development Workflow using and SDK" ===

This section can go away I think. We have the SDK manual and all it is here in the sdk-manual is a pointer to that manual.

=== Section 4.3 - "Modifying Source Code" ===

This section is conceptual by nature. Again, high-level procedures to support the concepts but I don't think they should be in the dev-manual. Both devtool and Quilt are covered in this section, and both regarding workflows designed to help the user modify source code. The actual need for this introductory stuff in 4.3 depends on really where the devtool and Quilt sections ultimately reside.

=== Section 4.3.1 - "Using devtool in Your Workflow" ===

These concepts belong in the ref-manual. Right now, this material is virtually duplicated in the sdk-manual. It should be one area and then any SDK-specific stuff can be captured in the sdk-manual. The section to put this in in the ref-manual would be under a "Concepts" heading and then maybe under something like "Workflows".

=== Section 4.3.2 - "Using Quilt in Your Workflow" ===

These concepts belong in the ref-manual in a new "Workflows" section bundled under some concepts heading. I can see creating an actual example that works and using it as a section in the dev-manual. The conceptual version from the ref-manual can reference into that example.

=== Section 4.3.3 - "Finding Temporary Source Code" ===

The reason this section is where it is was to support the Quilt workflow. If "Finding your Source Code" is a separate enough task to be in the dev-manual, then we can create a set of steps to walk a user through that give creation of some piece of software and maybe a kernel build. They seem to be different based on what you are building unless I am mis-understanding it.

=== Section 4.4 - "Image Development Using Toaster" ===

The reason this section exists is it was a development workflow and bundled in this overarching section. It is pretty much a placeholder and could be removed. There are ample areas where we can mention Toaster as a way of developing an image and then linking to the Toaster manual. If we have a workflow concepts section in the ref-manual, it can be mentioned there as a type of workflow.

=== Section 4.5 - "Using a Development Shell" ===

This is another development workflow. We could have a conceptual description in the appropriate area of the ref-manual that would point back to a specific example that is task-based in the dev-manual.

=== Section 4.6 - "Using a Development Python Shell" ===

This is the final development workflow section. Same stuff applies here as explained in the "Using a Development Shell" section.

Development Manual Rework

2017-06-23T22:27:58Z

Henry Bruce: /* Basic Skills for Application Developers */

= Intro =
This page will propose an outline for [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html Development Manual] refactoring work. See also [https://bugzilla.yoctoproject.org/show_bug.cgi?id=11630 bug #11630]

= Henry's Input =
I believe the manual lost its way as it became the dumping ground for useful (and often advanced) info that had nowhere else to land. As a result the size of the manual and top level topics became overwhelming to the new developer who had just graduated from the quick state guide. Currently this user is not served by the Yocto documentation, thus the creation of the wiki [[cookbook]]. Simple tasks such as adding an existing recipe to image and creating a recipe source should be the bread and butter of the development manual. If this approach makes the development master server too many master, we should dicuss re-structing all the docs the addition of a new one.

== Basic "How To" Topics ==
I plan to structure these, but just a plain list for the time being
=== Basic Skills for Systems Developers ===
* Deal with corporate proxies.
* Add a package to an image
* Understand the difference between a recipe and package
* Build a package by itself and why that's useful
* Find out what packages are created by a recipe
* Find out what files are in a package
* Find out what files are in an image
* Add an ssh server to an image (enable transferring of files to target)
* Anatomy of a recipe
* How to find recipes (layers.openembedded,org)
* Understand difference between machine and distro settings
* How to find and use the right BSP (machine) for your hardware
* Examples of distro features and where to set them.
* Understanding the task pipeline and executing individual tasks
* Understand devtool and how it simplifies your workflow
* Improve build speeds with shared downloads and shared state cache
* Generate and understand dependency graph
* Generate and understand bitbake environment
* Building an Extensible SDK
=== Basic Skills for Application Developers ===
* Understand intent and content of extensible SDK (eSDK) and why traditional SDK is no longer necessary
* Understand CROPS eSDK container solution
** Source code is edited outside of container using tools of your choice on host
**
* Install and run CROPS/eSDK
* Intro to devtool
* Make sure you're up to date
* Build the image (may not be the same as on your target) and typical options for flashing it
* Modify an existing recipe
** Understand how git is managing source code
** Build and deploy modified recipe
** Using devtool finish to create updated recipe and patches.
** Understand how this is done without updating upstream source code
* Add a new recipe
** Create recipe from source code
** Review recipe to see what's missing
** Repeat steps as for 'modify'

= Tracey's Input =
The way Henry described this yesterday, customers use the "getting started" guide to learn how to bring up a vanilla image, but then don't know how to approach their own custom situation. This should be a TRANSITION DOCUMENT - how to use YP once you have gotten your feet wet in a controlled situation. Feel free to delete this, I don't know how you folks work in the WIKI and I'm assuming this may not be it.

I would suggest organizing this in a couple of ways - and this might be better in a google doc, frankly.
1) by what people need to do to get started
a) where to start to support a platform
1. find a BSP and test it
b) how to add support for platform components
c) how to slim down an image size
1. Remove code automatically included
2. design your own BSP fine tuned to exclude extra code, unneeded support code
d) How to approach setting up a development environment
e) How use layers to create a flexible long term design, and recipes
d) How do you start a new project vs build off an existing one
e) How do you set up a team sharable development environment (sstate)
f) Here are some of the most common commands
g) How to deal with CVEs and code branches

2) By helping them collect the information they need in order to make set up decisions
Brent had a great idea to make this a SW Wizard that would step a new user through a series of questions and give them at least one solution on how to proceed.
a) what platform are you using
b) will you include an application as part of the image or will it run on the image
c) what is the configuration of the team (system developers, app developers, etc)
c) How to help them choose the right kernel
d) etc.

= Scott's Input =

Scott Rifenbark - This is a good idea. I was thinking of a similar way to deal with this with sort of a decision tree at the end of the YP Quick Start (flowchart for old-schoolers) that would lead the reader down a path based on information they have about what they want to get accomplished. Then, based on how they traverse the tree, we point them to areas they need to use in the manual.

== Notes for Chapter 1 of the dev-manual: ==

I re-wrote this introductory section to work more tightly with the introductory material of the ref-manual. Also, created a better "Other Information" section to lead the reader to various places of interest. In particular, I am leveraging off the Yocto Project Backgrounders that exist on the YP Website. These backgrounders are probably the best place for an overall introduction to what the YP is supposed to be and do for you.

== Notes for Chapter 3 of the dev-manual: ==

This chapter was mostly reference material. The entire chapter has been converted now to procedures by pulling out the reference stuff and placing it into the ref-manual. What is left in Chapter 3 are procedures related to setting up a shared YP development environment, submitting a defect, and submitting a change.

=== Section 3.1 - "Open Source Philosophy" ===

I moved this section to the ref-manual. It now is in a chapter called "The Yocto Project Development Environment". That new chapter in the ref-manual is really the old "Closer Look" chapter now with some YP development environment concepts placed on top in their own subsections.

=== Section 3.2 - "Using the Yocto Project in a Team Environment" ===

I have converted this section to a procedure. I need more information on this section from the team. Emails are out soliciting input.

=== Section 3.3 - "Source Repositories" ===

This section has been moved to the ref-manual under "The Yocto Project Development Environment" section. No procedures here except perhaps "Locating the Yocto Project Source Repositories".

=== Section 3.4 - "Terms" ===

This section has been moved to the ref-manual into the "Introduction" chapter. All links throughout the YP Documentation Manual set have been adjusted to work for the new location.

=== Section 3.5 - "Licensing" ===

I moved this section to the ref-manual under "The Yocto Project Development Environment". There are no possibilities of procedures in this section. It is pure reference material.

=== Section 3.6 - "Git" ===

The concepts and overview of how Git works is best suited for the ref-manual. I moved the concepts of Git to the ref-manual. In doing so, I decided to keep the two small examples in that section there just for completeness. However, I created a new section in the dev-manual called "Working With Git Repositories" that has pointed procedures that show the reader how to clone poky, check out a branch based on a poky branch name, and check out a branch based on a poky tag. Right now in the "Getting Set Up" section but will likely be moved as the new dev-manual gains more procedures.

=== Section 3.7 - "Workflows" ===

I have moved this section to "The Yocto Project Development Environment" section in the ref-manual. The section is concepts only.

=== Section 3.8 - "Submitting a Defect Against the Yocto Project" ===

I have reworked this section and it is done. It will live in the dev-manual.

=== Section 3.9 - "How to Submit a Change" ===

I have re-worked this section into a couple of procedures.

== Notes for Chapter 4 of the dev-manual: ==

This whole section is about various workflows. The information is a mix of concepts, high-level workflow steps, and various details associated with the flows. I think the entire chapter can pretty much be recast in some conceptual area of the ref-manual. Then, for each type of workflow, we can create a specific task that has some meaningful steps and real examples for use in the dev-manual. These sections in the dev-manual can be referenced from the associated concepts in the ref-manual.

=== Section 4.1 - "System Development Workflow" ===

Introductory material that will live with the subsections for concepts (in the ref-manual) or introductory area for grouped tasks in the dev-manual.

=== Section 4.1.1 - "Developing a Board Support Package (BSP)" ===

Although the steps in this section are technically procedural, the level is more conceptual. We should move this discussion to the ref-manual and group it under some sort of new "concepts" heading where conceptual information is discussed. That is one idea. Another, is to create a conceptual section in the BSP Guide to hold this information. One thing we have not discussed is the fate of the BSP Guide in regards to the restructuring of the dev-manual to a task-based document. In any case, the current "Developing a Board Support Package (BSP)" section in the dev-manual is out as it stands now. If we want to replace it in the dev-manual with something more concrete, then I suggest a real example that would be in the form of what is on this dated wiki page - https://wiki.yoctoproject.org/wiki/Transcript:_creating_one_generic_Atom_BSP_from_another.

=== Section 4.1.2 - "Modifying the Kernel" ===

The information in this section is very conceptual and does not belong in the dev-manual. Solution for this section is similar to what I suggested for the "Developing a Board Support Package (BSP)" section above. However, this is kernel related so perhaps moving this kernel information to the kernel-dev manual. Again, if we want to have some concrete example in the dev-manual that is based on the conceptual information here, then we create a real example and use that and tie it to specific tasks.

=== Section 4.2 - "Application Development Workflow using and SDK" ===

This section can go away I think. We have the SDK manual and all it is here in the sdk-manual is a pointer to that manual.

=== Section 4.3 - "Modifying Source Code" ===

This section is conceptual by nature. Again, high-level procedures to support the concepts but I don't think they should be in the dev-manual. Both devtool and Quilt are covered in this section, and both regarding workflows designed to help the user modify source code. The actual need for this introductory stuff in 4.3 depends on really where the devtool and Quilt sections ultimately reside.

=== Section 4.3.1 - "Using devtool in Your Workflow" ===

These concepts belong in the ref-manual. Right now, this material is virtually duplicated in the sdk-manual. It should be one area and then any SDK-specific stuff can be captured in the sdk-manual. The section to put this in in the ref-manual would be under a "Concepts" heading and then maybe under something like "Workflows".

=== Section 4.3.2 - "Using Quilt in Your Workflow" ===

These concepts belong in the ref-manual in a new "Workflows" section bundled under some concepts heading. I can see creating an actual example that works and using it as a section in the dev-manual. The conceptual version from the ref-manual can reference into that example.

=== Section 4.3.3 - "Finding Temporary Source Code" ===

The reason this section is where it is was to support the Quilt workflow. If "Finding your Source Code" is a separate enough task to be in the dev-manual, then we can create a set of steps to walk a user through that give creation of some piece of software and maybe a kernel build. They seem to be different based on what you are building unless I am mis-understanding it.

=== Section 4.4 - "Image Development Using Toaster" ===

The reason this section exists is it was a development workflow and bundled in this overarching section. It is pretty much a placeholder and could be removed. There are ample areas where we can mention Toaster as a way of developing an image and then linking to the Toaster manual. If we have a workflow concepts section in the ref-manual, it can be mentioned there as a type of workflow.

=== Section 4.5 - "Using a Development Shell" ===

This is another development workflow. We could have a conceptual description in the appropriate area of the ref-manual that would point back to a specific example that is task-based in the dev-manual.

=== Section 4.6 - "Using a Development Python Shell" ===

This is the final development workflow section. Same stuff applies here as explained in the "Using a Development Shell" section.

Development Manual Rework

2017-06-23T22:01:54Z

Henry Bruce: /* Basic "How To" Topics */

= Intro =
This page will propose an outline for [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html Development Manual] refactoring work. See also [https://bugzilla.yoctoproject.org/show_bug.cgi?id=11630 bug #11630]

= Henry's Input =
I believe the manual lost its way as it became the dumping ground for useful (and often advanced) info that had nowhere else to land. As a result the size of the manual and top level topics became overwhelming to the new developer who had just graduated from the quick state guide. Currently this user is not served by the Yocto documentation, thus the creation of the wiki [[cookbook]]. Simple tasks such as adding an existing recipe to image and creating a recipe source should be the bread and butter of the development manual. If this approach makes the development master server too many master, we should dicuss re-structing all the docs the addition of a new one.

== Basic "How To" Topics ==
I plan to structure these, but just a plain list for the time being
=== Basic Skills for Systems Developers ===
* Deal with corporate proxies.
* Add a package to an image
* Understand the difference between a recipe and package
* Build a package by itself and why that's useful
* Find out what packages are created by a recipe
* Find out what files are in a package
* Find out what files are in an image
* Add an ssh server to an image (enable transferring of files to target)
* Anatomy of a recipe
* How to find recipes (layers.openembedded,org)
* Understand difference between machine and distro settings
* How to find and use the right BSP (machine) for your hardware
* Examples of distro features and where to set them.
* Understanding the task pipeline and executing individual tasks
* Understand devtool and how it simplifies your workflow
* Improve build speeds with shared downloads and shared state cache
* Generate and understand dependency graph
* Generate and understand bitbake environment
* Building an Extensible SDK
=== Basic Skills for Application Developers ===

= Tracey's Input =
The way Henry described this yesterday, customers use the "getting started" guide to learn how to bring up a vanilla image, but then don't know how to approach their own custom situation. This should be a TRANSITION DOCUMENT - how to use YP once you have gotten your feet wet in a controlled situation. Feel free to delete this, I don't know how you folks work in the WIKI and I'm assuming this may not be it.

I would suggest organizing this in a couple of ways - and this might be better in a google doc, frankly.
1) by what people need to do to get started
a) where to start to support a platform
1. find a BSP and test it
b) how to add support for platform components
c) how to slim down an image size
1. Remove code automatically included
2. design your own BSP fine tuned to exclude extra code, unneeded support code
d) How to approach setting up a development environment
e) How use layers to create a flexible long term design, and recipes
d) How do you start a new project vs build off an existing one
e) How do you set up a team sharable development environment (sstate)
f) Here are some of the most common commands
g) How to deal with CVEs and code branches

2) By helping them collect the information they need in order to make set up decisions
Brent had a great idea to make this a SW Wizard that would step a new user through a series of questions and give them at least one solution on how to proceed.
a) what platform are you using
b) will you include an application as part of the image or will it run on the image
c) what is the configuration of the team (system developers, app developers, etc)
c) How to help them choose the right kernel
d) etc.

= Scott's Input =

Scott Rifenbark - This is a good idea. I was thinking of a similar way to deal with this with sort of a decision tree at the end of the YP Quick Start (flowchart for old-schoolers) that would lead the reader down a path based on information they have about what they want to get accomplished. Then, based on how they traverse the tree, we point them to areas they need to use in the manual.

== Notes for Chapter 1 of the dev-manual: ==

I re-wrote this introductory section to work more tightly with the introductory material of the ref-manual. Also, created a better "Other Information" section to lead the reader to various places of interest. In particular, I am leveraging off the Yocto Project Backgrounders that exist on the YP Website. These backgrounders are probably the best place for an overall introduction to what the YP is supposed to be and do for you.

== Notes for Chapter 3 of the dev-manual: ==

This chapter was mostly reference material. The entire chapter has been converted now to procedures by pulling out the reference stuff and placing it into the ref-manual. What is left in Chapter 3 are procedures related to setting up a shared YP development environment, submitting a defect, and submitting a change.

=== Section 3.1 - "Open Source Philosophy" ===

I moved this section to the ref-manual. It now is in a chapter called "The Yocto Project Development Environment". That new chapter in the ref-manual is really the old "Closer Look" chapter now with some YP development environment concepts placed on top in their own subsections.

=== Section 3.2 - "Using the Yocto Project in a Team Environment" ===

I have converted this section to a procedure. I need more information on this section from the team. Emails are out soliciting input.

=== Section 3.3 - "Source Repositories" ===

This section has been moved to the ref-manual under "The Yocto Project Development Environment" section. No procedures here except perhaps "Locating the Yocto Project Source Repositories".

=== Section 3.4 - "Terms" ===

This section has been moved to the ref-manual into the "Introduction" chapter. All links throughout the YP Documentation Manual set have been adjusted to work for the new location.

=== Section 3.5 - "Licensing" ===

I moved this section to the ref-manual under "The Yocto Project Development Environment". There are no possibilities of procedures in this section. It is pure reference material.

=== Section 3.6 - "Git" ===

The concepts and overview of how Git works is best suited for the ref-manual. I moved the concepts of Git to the ref-manual. In doing so, I decided to keep the two small examples in that section there just for completeness. However, I created a new section in the dev-manual called "Working With Git Repositories" that has pointed procedures that show the reader how to clone poky, check out a branch based on a poky branch name, and check out a branch based on a poky tag. Right now in the "Getting Set Up" section but will likely be moved as the new dev-manual gains more procedures.

=== Section 3.7 - "Workflows" ===

I have moved this section to "The Yocto Project Development Environment" section in the ref-manual. The section is concepts only.

=== Section 3.8 - "Submitting a Defect Against the Yocto Project" ===

I have reworked this section and it is done. It will live in the dev-manual.

=== Section 3.9 - "How to Submit a Change" ===

I have re-worked this section into a couple of procedures.

== Notes for Chapter 4 of the dev-manual: ==

This whole section is about various workflows. The information is a mix of concepts, high-level workflow steps, and various details associated with the flows. I think the entire chapter can pretty much be recast in some conceptual area of the ref-manual. Then, for each type of workflow, we can create a specific task that has some meaningful steps and real examples for use in the dev-manual. These sections in the dev-manual can be referenced from the associated concepts in the ref-manual.

=== Section 4.1 - "System Development Workflow" ===

Introductory material that will live with the subsections for concepts (in the ref-manual) or introductory area for grouped tasks in the dev-manual.

=== Section 4.1.1 - "Developing a Board Support Package (BSP)" ===

Although the steps in this section are technically procedural, the level is more conceptual. We should move this discussion to the ref-manual and group it under some sort of new "concepts" heading where conceptual information is discussed. That is one idea. Another, is to create a conceptual section in the BSP Guide to hold this information. One thing we have not discussed is the fate of the BSP Guide in regards to the restructuring of the dev-manual to a task-based document. In any case, the current "Developing a Board Support Package (BSP)" section in the dev-manual is out as it stands now. If we want to replace it in the dev-manual with something more concrete, then I suggest a real example that would be in the form of what is on this dated wiki page - https://wiki.yoctoproject.org/wiki/Transcript:_creating_one_generic_Atom_BSP_from_another.

=== Section 4.1.2 - "Modifying the Kernel" ===

The information in this section is very conceptual and does not belong in the dev-manual. Solution for this section is similar to what I suggested for the "Developing a Board Support Package (BSP)" section above. However, this is kernel related so perhaps moving this kernel information to the kernel-dev manual. Again, if we want to have some concrete example in the dev-manual that is based on the conceptual information here, then we create a real example and use that and tie it to specific tasks.

=== Section 4.2 - "Application Development Workflow using and SDK" ===

This section can go away I think. We have the SDK manual and all it is here in the sdk-manual is a pointer to that manual.

=== Section 4.3 - "Modifying Source Code" ===

This section is conceptual by nature. Again, high-level procedures to support the concepts but I don't think they should be in the dev-manual. Both devtool and Quilt are covered in this section, and both regarding workflows designed to help the user modify source code. The actual need for this introductory stuff in 4.3 depends on really where the devtool and Quilt sections ultimately reside.

=== Section 4.3.1 - "Using devtool in Your Workflow" ===

These concepts belong in the ref-manual. Right now, this material is virtually duplicated in the sdk-manual. It should be one area and then any SDK-specific stuff can be captured in the sdk-manual. The section to put this in in the ref-manual would be under a "Concepts" heading and then maybe under something like "Workflows".

=== Section 4.3.2 - "Using Quilt in Your Workflow" ===

These concepts belong in the ref-manual in a new "Workflows" section bundled under some concepts heading. I can see creating an actual example that works and using it as a section in the dev-manual. The conceptual version from the ref-manual can reference into that example.

=== Section 4.3.3 - "Finding Temporary Source Code" ===

The reason this section is where it is was to support the Quilt workflow. If "Finding your Source Code" is a separate enough task to be in the dev-manual, then we can create a set of steps to walk a user through that give creation of some piece of software and maybe a kernel build. They seem to be different based on what you are building unless I am mis-understanding it.

=== Section 4.4 - "Image Development Using Toaster" ===

The reason this section exists is it was a development workflow and bundled in this overarching section. It is pretty much a placeholder and could be removed. There are ample areas where we can mention Toaster as a way of developing an image and then linking to the Toaster manual. If we have a workflow concepts section in the ref-manual, it can be mentioned there as a type of workflow.

=== Section 4.5 - "Using a Development Shell" ===

This is another development workflow. We could have a conceptual description in the appropriate area of the ref-manual that would point back to a specific example that is task-based in the dev-manual.

=== Section 4.6 - "Using a Development Python Shell" ===

This is the final development workflow section. Same stuff applies here as explained in the "Using a Development Shell" section.

Development Manual Rework

2017-06-20T01:00:57Z

Henry Bruce:

= Intro =
This page will propose an outline for [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html Development Manual] refactoring work. See also [https://bugzilla.yoctoproject.org/show_bug.cgi?id=11630 bug #11630]

= Henry's Input =
I believe the manual lost its way as it became the dumping ground for useful (and often advanced) info that had nowhere else to land. As a result the size of the manual and top level topics became overwhelming to the new developer who had just graduated from the quick state guide. Currently this user is not served by the Yocto documentation, thus the creation of the wiki [[cookbook]]. Simple tasks such as adding an existing recipe to image and creating a recipe source should be the bread and butter of the development manual. If this approach makes the development master server too many master, we should dicuss re-structing all the docs the addition of a new one.

== Basic "How To" Topics ==
I plan to structure these, but just a plain list for the time being
* Understand the difference between a recipe and package
* Add a package to an image
* Build a package by itself and why that's useful
* Find out what packages are created by a recipe
* Find out what files are in a package
* Find out what files are in an image
* Add an ssh server to an image (enable transferring of files to target)
* Deal with corporate proxies.

More to come

= Tracey's Input =
The way Henry described this yesterday, customers use the "getting started" guide to learn how to bring up a vanilla image, but then don't know how to approach their own custom situation. This should be a TRANSITION DOCUMENT - how to use YP once you have gotten your feet wet in a controlled situation. Feel free to delete this, I don't know how you folks work in the WIKI and I'm assuming this may not be it.

I would suggest organizing this in a couple of ways - and this might be better in a google doc, frankly.
1) by what people need to do to get started
a) where to start to support a platform
1. find a BSP and test it
b) how to add support for platform components
c) how to slim down an image size
1. Remove code automatically included
2. design your own BSP fine tuned to exclude extra code, unneeded support code
d) How to approach setting up a development environment
e) How use layers to create a flexible long term design, and recipes
d) How do you start a new project vs build off an existing one
e) How do you set up a team sharable development environment (sstate)
f) Here are some of the most common commands
g) How to deal with CVEs and code branches

2) By helping them collect the information they need in order to make set up decisions
Brent had a great idea to make this a SW Wizard that would step a new user through a series of questions and give them at least one solution on how to proceed.
a) what platform are you using
b) will you include an application as part of the image or will it run on the image
c) what is the configuration of the team (system developers, app developers, etc)
c) How to help them choose the right kernel
d) etc.

= Scott's Input =

Scott Rifenbark - This is a good idea. I was thinking of a similar way to deal with this with sort of a decision tree at the end of the YP Quick Start (flowchart for old-schoolers) that would lead the reader down a path based on information they have about what they want to get accomplished. Then, based on how they traverse the tree, we point them to areas they need to use in the manual.

== Notes for Chapter 3 of the dev-manual: ==

This chapter is mostly reference material. The only stuff that I am going to convert to procedures is the various subsections on how to submit a change. Ultimately, the chapter will disappear in the dev-manual as I move stuff out of it. What does remain will be located somewhere in the dev-manual as tasks. I am not sure just where yet.

=== Section 3.1 - "Open Source Philosophy" ===

I moved this section to the ref-manual. It now is in a chapter called "The Yocto Project Development Environment". That new chapter in the ref-manual is really the old "Closer Look" chapter now with some YP development environment concepts placed on top in their own subsections.

=== Section 3.2 - "Using the Yocto Project in a Team Environment" ===

This section consists of introductory information for team use and provides a bunch of tips and best practices. I have converted this section to a procedure. For now, you can see the first draft at http://www.yoctoproject.org/docs/scott_temp/dev-manual/dev-manual.html#usingpoky-changes-collaborate, which is in a temporary holding area where the published HTML YP documents live. I need more information on this section from the team. Emails are out soliciting input.

=== Section 3.3 - "Source Repositories" ===

This section has been moved to the ref-manual under "The Yocto Project Development Environment" section. No procedures here except perhaps "Locating the Yocto Project Source Repositories".

=== Section 3.4 - "Terms" ===

This section has been moved to the ref-manual into the "Introduction" chapter. All links throughout the YP Documentation Manual set have been adjusted to work for the new location.

=== Section 3.5 - "Licensing" ===

I moved this section to the ref-manual under "The Yocto Project Development Environment". There are no possibilities of procedures in this section. It is pure reference material.

=== Section 3.6 - "Git" ===

The concepts and overview of how Git works is best suited for the ref-manual. I moved the concepts of Git to the ref-manual. In doing so, I decided to keep the two small examples in that section there just for completeness. However, I created a new section in the dev-manual called "Working With Git Repositories" that has pointed procedures that show the reader how to clone poky, check out a branch based on a poky branch name, and check out a branch based on a poky tag. Right now in the "Getting Set Up" section but will likely be moved as the new dev-manual gains more procedures.

=== Section 3.7 - "Workflows" ===

I have moved this section to "The Yocto Project Development Environment" section in the ref-manual. The section is concepts only.

=== Section 3.8 - "Submitting a Defect Against the Yocto Project" ===

I have reworked this section and it is done. It will live in the dev-manual.

=== Section 3.9 - "How to Submit a Change" ===

This section has a lot of potential for "how-to" procedures. I think the whole section can be re-done to be a procedure that walks the reader through the process. I can create some conceptual information that could live in the ref-manual and then have it point back to the processes here in the dev-manual.

== Notes for Chapter 4 of the dev-manual: ==

This whole section is about various workflows. The information is a mix of concepts, high-level workflow steps, and various details associated with the flows. I think the entire chapter can pretty much be recast in some conceptual area of the ref-manual. Then, for each type of workflow, we can create a specific task that has some meaningful steps and real examples for use in the dev-manual. These sections in the dev-manual can be referenced from the associated concepts in the ref-manual.

=== Section 4.1 - "System Development Workflow" ===

Introductory material that will live with the subsections for concepts (in the ref-manual) or introductory area for grouped tasks in the dev-manual.

=== Section 4.1.1 - "Developing a Board Support Package (BSP)" ===

Although the steps in this section are technically procedural, the level is more conceptual. We should move this discussion to the ref-manual and group it under some sort of new "concepts" heading where conceptual information is discussed. That is one idea. Another, is to create a conceptual section in the BSP Guide to hold this information. One thing we have not discussed is the fate of the BSP Guide in regards to the restructuring of the dev-manual to a task-based document. In any case, the current "Developing a Board Support Package (BSP)" section in the dev-manual is out as it stands now. If we want to replace it in the dev-manual with something more concrete, then I suggest a real example that would be in the form of what is on this dated wiki page - https://wiki.yoctoproject.org/wiki/Transcript:_creating_one_generic_Atom_BSP_from_another.

=== Section 4.1.2 - "Modifying the Kernel" ===

The information in this section is very conceptual and does not belong in the dev-manual. Solution for this section is similar to what I suggested for the "Developing a Board Support Package (BSP)" section above. However, this is kernel related so perhaps moving this kernel information to the kernel-dev manual. Again, if we want to have some concrete example in the dev-manual that is based on the conceptual information here, then we create a real example and use that and tie it to specific tasks.

=== Section 4.2 - "Application Development Workflow using and SDK" ===

This section can go away I think. We have the SDK manual and all it is here in the sdk-manual is a pointer to that manual.

=== Section 4.3 - "Modifying Source Code" ===

This section is conceptual by nature. Again, high-level procedures to support the concepts but I don't think they should be in the dev-manual. Both devtool and Quilt are covered in this section, and both regarding workflows designed to help the user modify source code. The actual need for this introductory stuff in 4.3 depends on really where the devtool and Quilt sections ultimately reside.

=== Section 4.3.1 - "Using devtool in Your Workflow" ===

These concepts belong in the ref-manual. Right now, this material is virtually duplicated in the sdk-manual. It should be one area and then any SDK-specific stuff can be captured in the sdk-manual. The section to put this in in the ref-manual would be under a "Concepts" heading and then maybe under something like "Workflows".

=== Section 4.3.2 - "Using Quilt in Your Workflow" ===

These concepts belong in the ref-manual in a new "Workflows" section bundled under some concepts heading. I can see creating an actual example that works and using it as a section in the dev-manual. The conceptual version from the ref-manual can reference into that example.

=== Section 4.3.3 - "Finding Temporary Source Code" ===

The reason this section is where it is was to support the Quilt workflow. If "Finding your Source Code" is a separate enough task to be in the dev-manual, then we can create a set of steps to walk a user through that give creation of some piece of software and maybe a kernel build. They seem to be different based on what you are building unless I am mis-understanding it.

=== Section 4.4 - "Image Development Using Toaster" ===

The reason this section exists is it was a development workflow and bundled in this overarching section. It is pretty much a placeholder and could be removed. There are ample areas where we can mention Toaster as a way of developing an image and then linking to the Toaster manual. If we have a workflow concepts section in the ref-manual, it can be mentioned there as a type of workflow.

=== Section 4.5 - "Using a Development Shell" ===

This is another development workflow. We could have a conceptual description in the appropriate area of the ref-manual that would point back to a specific example that is task-based in the dev-manual.

=== Section 4.6 - "Using a Development Python Shell" ===

This is the final development workflow section. Same stuff applies here as explained in the "Using a Development Shell" section.

Developer Workflow Improvements

2017-06-17T01:05:05Z

Henry Bruce: /* Layer Maintenance */

Back to Yocto Project [[Main Page]]
= Introduction =
The page describes how new tools such as [https://www.gitbook.com/book/dnplas/yocto-project-crops/details Cross Platform Support] (CROPS), [http://www.yoctoproject.org/docs/current/sdk-manual/sdk-manual.html#sdk-extensible-sdk-intro Extensible SDK] (eSDK), [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#dev-modifying-source-code devtool] and [https://www.yoctoproject.org/docs/current/toaster-manual/toaster-manual.html#toaster-manual-intro Toaster] make it easier to get the best out of Yocto. To benefit from this article the reader must be comfortable with the basics of the Yocto Project (i.e. has gone through [http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html Quick Start Guide] and knows how to add packages to an image).

We envisage two distinct workflows, layer maintenance and application development.

== Layer Maintenance ==
* This workflow is mainly bug fixing recipes and upgrading to never versions. Updated recipes are then submitted to layer maintainers. It also includes building images that exercise the recipes under test.
* Typically this is done in the distro maintainer (i.e. bitbake) environment, but the extensible SDK can be a better option as it enables an easy to configure common development environment.

== Application Development ==
* Poky or Toaster is used to generate a minimal eSDK installer.
* A developer uses resulting eSDK to create/modify an application by using devtool (iterative process).
* Once application is complete its recipe is added to a layer and Toaster is used to build an image that includes it

= Workflow Vision =
This section calls out the workflows in more detail, describing specific use of tools in each step.

The [[Developer Workflow Improvements#Instructions for Workflow in Current Form | set-up instructions]] at the end of this page represent the "what can currently be done" version of the workflows.

== Build OS Image ==
# On OS of your choice start containerized Poky or Toaster instance.
# If running Poky container on Windows or Mac you must also start the samba container so that host OS has access to files in the container.
# Add new layers and packages as required to create OS on which applications will run
# Build image and validate

== Creating Extensible SDK ==
# Build image as above
# Then build Extensible SDK
# Publish components required for app development
## eSDK installer
## eSDK updater files
## shared state

== Application Development ==
=== One time set-up on OS of your choice ===
# Create docker volume so that files fetched and created in the container are visible to host OS.
# Use extensible SDK container to install eSDK to volume.
# Development can also be done on the command line.
# Optionally install Eclipse and CROPS plug-in that
=== App development ===
# Start extensible SDK container so you can run devtool from command line or invoke it from Eclipse plug-in.
# Start samba container so that files are visible to host OS
# Use '''devtool''' to 'add' application project from its URL. This pulls down source code, tries to work out dependent packages and generates a recipe
# Although generated recipe will likely build, it may need some tweaks to some of the tasks (e.g. configure, compile, install)
# Once app is building, iteratvely develop using devtool (or Eclipse) to deploy and test on target
# Build new image including new layer and recipe and validate

== Layer Maintenance ==
# Build image as above
# Use '''devtool modify''' to update recipe
# Iterative development as per application development.
# Build image and validate
# Publish updated recipe

= Instructions for Workflow in Current Form =
If you are inside a corporate firewall, install and configure the [https://github.com/crops/chameleonsocks Chameleonsocks] container based proxy solution.

You have choice between using the Yocto Project tools on the command line with the Poky container or by using the Toaster web interface. Toaster is easier to get going but most needs more work to get the best of out interface (see section below), so these instructions will assume use of the command line.

== Build OS image ==
We recommend that you first [https://github.com/crops/docker-win-mac-docs/wiki install docker] on your OS of choice and then follow instructions to set up [https://github.com/crops/poky-container/blob/master/README.md Poky] or [https://github.com/crops/toaster-container/blob/master/README.md Toaster] containers. However, these instructions will work for Poky or Toaster on Linux directly but you're on your own for installation and configuration.

Now add layers and configure so you can build your image.

== Build and Publish eSDK ==
This section is not part of the developer workflow but documents how the SDK components are generated and published.
This step creates a small installer that is made available to the developer. You must have access to an http server so you can publish SDK update files. 
Building and publishing the SDK takes a number of steps.
=== Configure ===
Set the following variables in local.conf
:<tt>SDK_EXT_TYPE = "minimal"</tt> ensure a minimal SDK build. This produces a small installer and enable components to de downloaded as they are installed
: SDK_UPDATE_URL points eSDK to URL that contains updates
:Optionally set <tt>SDK_INCLUDE_PKGDATA = "1"</tt> to help eSDK users by pre-building all packages in all layers (i.e. not just packages included in the image). The installer remains small as build data is made available via sstate mirror. Downside is that eSDK build will take much longer.

Here are some example settings
SDK_EXT_TYPE = "minimal"
SDK_UPDATE_URL = "http://my.server.com/path/to/esdk-update"
SDK_INCLUDE_PKGDATA = "1"

The URL for the eSDK sstate mirror is not included in the above settings as it would conflict with the mirror used by the main build system. To handle this conflict, the value in put into a separate conf file <tt>conf/sdk-extra.conf</tt>
SSTATE_MIRRORS = "file://.* http://my.server.com/path/to/sstate-cache/PATH"
Note that if you are using Toasrter, you don't have access to <tt>conf/sdk-extra.conf</tt>, so you have to set variables via the U/I. This means that Toaster cannot have an sstate mirror for the OS build and set the eSDK's as follows:
SDK_LOCAL_CONF_WHITELIST = "SSTATE_MIRRORS"
SSTATE_MIRRORS = "file://.* http://my.server.com/path/to/sstate-cache/PATH"

=== Build ===
Assuming image name is my-image, build as follows from command line
bitbake my-image -c populate_sdk_ext
If using Toaster, build <tt>my-image:populate_sdk_ext</tt>.

=== Publish SDK Updater ===
The eSDK allows web updates, so you have to ensure they appeat at the URK specified by SDK_UPDATE_URL. Assuming /var/www/html/path/to is served up as http://my.server.com/path/to, publish as follows
oe-publish-sdk tmp/deploy/sdk/distro-image-toolchain-ext-ver.sh /var/www/html/path/to/esdk-update

=== Publish SSTATE ===
cp -r build/sstate-cache /var/www/html/path/to/sstate-cache

=== Publish installer ===
Copy installer to web server and give this URL to your app developers.
cp tmp/deploy/sdk/distro-image-toolchain-ext-ver.sh /var/www/html/path/to/sdk-installer.sh

== Application Development ==
=== Install and Use eSDK ===
* This assumes you have docker already installed as per Toaster instructions. No other dependencies are required (that the beauty of containers).
* Create folder for esdk bind mount where output from container will appear, say /path/to/esdk/workdir
mkdir /path/to/esdk/workdir
* Run container, pointing to URL of eSDK minimal installer. You must do this every time you want to use eSDK. In this example I'm suing an eSDk installer I created that is available at https://downloads.yoctoproject.org/tools/support/workflow/poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-2.2.sh
docker run --rm -it -v /path/to/esdk-workdir:/workdir crops/extsdk-container --url https://downloads.yoctoproject.org/tools/support/workflow/poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-2.2.sh
If you see the error '''ERROR: The extensible sdk cannot be installed as root.''' it means that the folder /path/to/esdk-workdir does not exist
* You now be in the eSDK container shell.
** For more details on eSDK container see https://github.com/crops/extsdk-container.
* Now follow instructions from [https://wiki.yoctoproject.org/wiki/Cookbook:Example:Creating_and_updating_recipes_with_devtool Yocto Project cookbook] on building an app.
* Recipe will be available in /path/to/esdk/workdir/workspace/recipes/bbexample/

= Missing Features =
These are the features currently missing from the above workflow vision. Think of these as stories in an Agile development environment, rather than bugs. Where possible these are linked to bugzilla entries.
* Toaster: Base configuration (e.g. define what was Ostro in single configuration file) [https://bugzilla.yoctoproject.org/show_bug.cgi?id=9588 [9588]]
* Toaster: Simple generation of eSDK installer in Toaster. [https://bugzilla.yoctoproject.org/show_bug.cgi?id=9297 [9297]]
* Toaster: Layer Maintenance workflow. Requires "local layers" re-design. No bugzilla entry, but see [https://www.youtube.com/watch?v=z5wVjBwJDzY design proposal video]
* Toaster: Publish layer to Layer Index from Toaster [https://bugzilla.yoctoproject.org/show_bug.cgi?id=9895 [9895]]
* CROPS: Application builder (but running eSDK in a container gets us close). [https://bugzilla.yoctoproject.org/show_bug.cgi?id=10119 [10119]] [https://bugzilla.yoctoproject.org/show_bug.cgi?id=4209 [4209]]
* CROPS: Eclipse integration. ''In progress. No bugzilla entry.''

Developer Workflow Improvements

2017-06-17T01:01:40Z

Henry Bruce: /* Layer Maintenance */

Back to Yocto Project [[Main Page]]
= Introduction =
The page describes how new tools such as [https://www.gitbook.com/book/dnplas/yocto-project-crops/details Cross Platform Support] (CROPS), [http://www.yoctoproject.org/docs/current/sdk-manual/sdk-manual.html#sdk-extensible-sdk-intro Extensible SDK] (eSDK), [http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#dev-modifying-source-code devtool] and [https://www.yoctoproject.org/docs/current/toaster-manual/toaster-manual.html#toaster-manual-intro Toaster] make it easier to get the best out of Yocto. To benefit from this article the reader must be comfortable with the basics of the Yocto Project (i.e. has gone through [http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html Quick Start Guide] and knows how to add packages to an image).

We envisage two distinct workflows, layer maintenance and application development.

== Layer Maintenance ==
* This tasks is mainly bug fixing recipes and upgrading to never versions. It also includes building images that exercise the layers under test
* Typically this is done in the distro maintainer (i.e. bitbake) environment, but if no extra layers are to be added then the extensible SDK is a better option as it ensures a common development environment.

== Application Development ==
* Poky or Toaster is used to generate a minimal eSDK installer.
* A developer uses resulting eSDK to create/modify an application by using devtool (iterative process).
* Once application is complete its recipe is added to a layer and Toaster is used to build an image that includes it

= Workflow Vision =
This section calls out the workflows in more detail, describing specific use of tools in each step.

The [[Developer Workflow Improvements#Instructions for Workflow in Current Form | set-up instructions]] at the end of this page represent the "what can currently be done" version of the workflows.

== Build OS Image ==
# On OS of your choice start containerized Poky or Toaster instance.
# If running Poky container on Windows or Mac you must also start the samba container so that host OS has access to files in the container.
# Add new layers and packages as required to create OS on which applications will run
# Build image and validate

== Creating Extensible SDK ==
# Build image as above
# Then build Extensible SDK
# Publish components required for app development
## eSDK installer
## eSDK updater files
## shared state

== Application Development ==
=== One time set-up on OS of your choice ===
# Create docker volume so that files fetched and created in the container are visible to host OS.
# Use extensible SDK container to install eSDK to volume.
# Development can also be done on the command line.
# Optionally install Eclipse and CROPS plug-in that
=== App development ===
# Start extensible SDK container so you can run devtool from command line or invoke it from Eclipse plug-in.
# Start samba container so that files are visible to host OS
# Use '''devtool''' to 'add' application project from its URL. This pulls down source code, tries to work out dependent packages and generates a recipe
# Although generated recipe will likely build, it may need some tweaks to some of the tasks (e.g. configure, compile, install)
# Once app is building, iteratvely develop using devtool (or Eclipse) to deploy and test on target
# Build new image including new layer and recipe and validate

== Layer Maintenance ==
# Build image as above
# Use '''devtool modify''' to update recipe
# Iterative development as per application development.
# Build image and validate
# Publish updated recipe

= Instructions for Workflow in Current Form =
If you are inside a corporate firewall, install and configure the [https://github.com/crops/chameleonsocks Chameleonsocks] container based proxy solution.

You have choice between using the Yocto Project tools on the command line with the Poky container or by using the Toaster web interface. Toaster is easier to get going but most needs more work to get the best of out interface (see section below), so these instructions will assume use of the command line.

== Build OS image ==
We recommend that you first [https://github.com/crops/docker-win-mac-docs/wiki install docker] on your OS of choice and then follow instructions to set up [https://github.com/crops/poky-container/blob/master/README.md Poky] or [https://github.com/crops/toaster-container/blob/master/README.md Toaster] containers. However, these instructions will work for Poky or Toaster on Linux directly but you're on your own for installation and configuration.

Now add layers and configure so you can build your image.

== Build and Publish eSDK ==
This section is not part of the developer workflow but documents how the SDK components are generated and published.
This step creates a small installer that is made available to the developer. You must have access to an http server so you can publish SDK update files. 
Building and publishing the SDK takes a number of steps.
=== Configure ===
Set the following variables in local.conf
:<tt>SDK_EXT_TYPE = "minimal"</tt> ensure a minimal SDK build. This produces a small installer and enable components to de downloaded as they are installed
: SDK_UPDATE_URL points eSDK to URL that contains updates
:Optionally set <tt>SDK_INCLUDE_PKGDATA = "1"</tt> to help eSDK users by pre-building all packages in all layers (i.e. not just packages included in the image). The installer remains small as build data is made available via sstate mirror. Downside is that eSDK build will take much longer.

Here are some example settings
SDK_EXT_TYPE = "minimal"
SDK_UPDATE_URL = "http://my.server.com/path/to/esdk-update"
SDK_INCLUDE_PKGDATA = "1"

The URL for the eSDK sstate mirror is not included in the above settings as it would conflict with the mirror used by the main build system. To handle this conflict, the value in put into a separate conf file <tt>conf/sdk-extra.conf</tt>
SSTATE_MIRRORS = "file://.* http://my.server.com/path/to/sstate-cache/PATH"
Note that if you are using Toasrter, you don't have access to <tt>conf/sdk-extra.conf</tt>, so you have to set variables via the U/I. This means that Toaster cannot have an sstate mirror for the OS build and set the eSDK's as follows:
SDK_LOCAL_CONF_WHITELIST = "SSTATE_MIRRORS"
SSTATE_MIRRORS = "file://.* http://my.server.com/path/to/sstate-cache/PATH"

=== Build ===
Assuming image name is my-image, build as follows from command line
bitbake my-image -c populate_sdk_ext
If using Toaster, build <tt>my-image:populate_sdk_ext</tt>.

=== Publish SDK Updater ===
The eSDK allows web updates, so you have to ensure they appeat at the URK specified by SDK_UPDATE_URL. Assuming /var/www/html/path/to is served up as http://my.server.com/path/to, publish as follows
oe-publish-sdk tmp/deploy/sdk/distro-image-toolchain-ext-ver.sh /var/www/html/path/to/esdk-update

=== Publish SSTATE ===
cp -r build/sstate-cache /var/www/html/path/to/sstate-cache

=== Publish installer ===
Copy installer to web server and give this URL to your app developers.
cp tmp/deploy/sdk/distro-image-toolchain-ext-ver.sh /var/www/html/path/to/sdk-installer.sh

== Application Development ==
=== Install and Use eSDK ===
* This assumes you have docker already installed as per Toaster instructions. No other dependencies are required (that the beauty of containers).
* Create folder for esdk bind mount where output from container will appear, say /path/to/esdk/workdir
mkdir /path/to/esdk/workdir
* Run container, pointing to URL of eSDK minimal installer. You must do this every time you want to use eSDK. In this example I'm suing an eSDk installer I created that is available at https://downloads.yoctoproject.org/tools/support/workflow/poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-2.2.sh
docker run --rm -it -v /path/to/esdk-workdir:/workdir crops/extsdk-container --url https://downloads.yoctoproject.org/tools/support/workflow/poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-2.2.sh
If you see the error '''ERROR: The extensible sdk cannot be installed as root.''' it means that the folder /path/to/esdk-workdir does not exist
* You now be in the eSDK container shell.
** For more details on eSDK container see https://github.com/crops/extsdk-container.
* Now follow instructions from [https://wiki.yoctoproject.org/wiki/Cookbook:Example:Creating_and_updating_recipes_with_devtool Yocto Project cookbook] on building an app.
* Recipe will be available in /path/to/esdk/workdir/workspace/recipes/bbexample/

= Missing Features =
These are the features currently missing from the above workflow vision. Think of these as stories in an Agile development environment, rather than bugs. Where possible these are linked to bugzilla entries.
* Toaster: Base configuration (e.g. define what was Ostro in single configuration file) [https://bugzilla.yoctoproject.org/show_bug.cgi?id=9588 [9588]]
* Toaster: Simple generation of eSDK installer in Toaster. [https://bugzilla.yoctoproject.org/show_bug.cgi?id=9297 [9297]]
* Toaster: Layer Maintenance workflow. Requires "local layers" re-design. No bugzilla entry, but see [https://www.youtube.com/watch?v=z5wVjBwJDzY design proposal video]
* Toaster: Publish layer to Layer Index from Toaster [https://bugzilla.yoctoproject.org/show_bug.cgi?id=9895 [9895]]
* CROPS: Application builder (but running eSDK in a container gets us close). [https://bugzilla.yoctoproject.org/show_bug.cgi?id=10119 [10119]] [https://bugzilla.yoctoproject.org/show_bug.cgi?id=4209 [4209]]
* CROPS: Eclipse integration. ''In progress. No bugzilla entry.''

TipsAndTricks/EnablingAPackageFeed

2017-06-16T07:20:29Z

Henry Bruce: /* Further Reading */

== Caveat ==
This article is based on work done a few years ago with fido (1.8) and only covers using smart with rpm. Needs some testing and extended to cover using dnf.

Instructions assume you have had at least one successful build of your target image.

== Select Your Package Format ==
package format is set via variable PACKAGE_CLASSES, typically in local.conf. Look for line like
PACKAGE_CLASSES ?= "package_rpm"
This means you're using rpm.

== Know Your Package Architectures ==
Your OS image will be comprised of a number of different package architectures.
=== From the build system view ===
The package feed needs to know what they are so look in tmp/deploy/rpm
$ ls tmp/deploy/rpm/
all core2_32 edison x86_64_nativesdk
You can exclude anything starting with x86_64. This means the architectures on the target are: <tt>all core2_32 edison</tt>
=== From target system view ===
$ rpm -qq
look at the endings of the various packages. For example , if you build for qemux86-64 the suffixes will be
core2_64
qemux86_64
all
so those are the architectures supported on the target.

== Select Your Package Feed URL ==
Is this example we'll use <tt>http://my-server.com/repo</tt>

== Add Package Manager to Build ==
Smart package manager is in oe-core, so there is no need to add any extra layers. You do, however need to tell the build system to include pkg management on the target, since excluding package management saves space, it is not enabled by default.

EXTRA_IMAGE_FEATURES += " package-management "

On releases prior to jethro, you *may* need to also do
IMAGE_INSTALL_append=" python-smartrpm "

== Configure Package Feed in Build ==
Now you have the information to create a package feed, ideally you add details in your distro conf file. If you are not using your own distro, you can set the following in local.conf. Note, if you only want to configure your channels on the target and not have any defaults, these definitions can be excluded.
PACKAGE_FEED_URIS = "http://my-server.com/repo"
PACKAGE_FEED_BASE_PATHS = "rpm"
PACKAGE_FEED_ARCHS = "all edison core2_32"

== Create Package Feed ==
* Re-build image so it includes smart and the package feed info
** Also, if you find you need an additional recipe not included in the image e.g. gawk, you can do bitbake gawk and it's packages will be available too.
*** this can be done iteratively
* Update repo package indices (this step can be excluded in later versions of Yocto)
bitbake package-index
* Copy packages to server. This sample script assumes files are served from /var/www/html/repo
** We exclude x86_64 as those are host side (aka native) rpms
<pre>
rsync -r -u --exclude 'x86_64*' tmp/deploy/rpm/* /var/www/html/repo
</pre>

== Run Package Manager on Target ==
* Make sure network is up and sync with package feed.
smart update
* Install my-package
smart search my-package
smart install my-package

== Recover Package Feed Config On Target ==
Smart does not have a config file, so config must be done via command line. This script re-creates the config set by distro in example above
<pre>
feed_url="http://my-server.com/repo"
platform="my-platform"
repo_name="my-repo"
for arch in all edison core2_32; do
smart channel --add $platform-$arch type=rpm-md name="$repo_name" baseurl=$feed_url/$arch -y
done
</pre>
Rebuild the cache with the new channels added
$ smart update
This will result in a set of feeds you can see with
$ smart channel --show

== Quick and Dirty Feed Serving ==
If you don't have a web server already serving up a directory and don't feel like going through the effort to do that you can do the following.
* Assume your feed is /opt/feed
* apt-get/dnf install python-twistd
* run twistd on the directory
$ twistd -n web --path /opt/feed -p 5678
* This way you can serve diff feeds on diff ports easily

== Further Reading ==
* [http://labix.org/smart/user-guide Smart online user guide]
* [https://www.fujitsu.com/jp/group/fct/documents/events/2016/A_Smart_Way_to_Manage_Packages_in_Yocto_Project.pdf Fujitsu ELC 2016 Presentation]

== Future Work ==
* dnf
* httpd
* feed-stability

Articles/Battles with Cockpit

2017-06-16T06:02:20Z

Henry Bruce:

I had the task of creating a recipe for [http://cockpit-project.org/ Cockpit], a Linux platform configuration tool. A key dependency is [https://github.com/performancecopilot/pcp Performance Co-Pilot], a system-level performance monitoring and management tool. What didn't help is that I knew nothing about the project itself. I've been working with Yocto for almost 3 years. I'm getting better, but am still learning, as this experience has shown me. Here is my story.

I started by using devtool's recipe generation feature.
$ devtool add devtool add pcp https://github.com/performancecopilot/pcp.git
It gave me a recipe, let's take a look
<pre>
# Recipe created by recipetool
# This is the basis of a recipe and may need further editing in order to be fully functional.
# (Feel free to remove these comments when editing.)

SUMMARY = "System-level performance monitoring and performance management"
# WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is
# your responsibility to verify that the values are complete and correct.
#
# NOTE: multiple licenses have been detected; they have been separated with &
# in the LICENSE value for now since it is a reasonable assumption that all
# of the licenses apply. If instead there is a choice between the multiple
# licenses then you should change the value to separate the licenses with |
# instead of &. If there is any doubt, check the accompanying documentation
# to determine which situation is applicable.
#
# The following license files were not able to be identified and are
# represented as "Unknown" below, you will need to check them yourself:
# COPYING
# man/html/qwtlicense.html
# debian/copyright
# build/mac/installer-resources/License.html
#
# NOTE: spec file indicates the license may be "GPLv2+ and LGPLv2.1+ and CC-BY"
LICENSE = "Unknown"
LIC_FILES_CHKSUM = "file://COPYING;md5=37ab75b580d5aad4ada04260efa3702f \
file://man/html/qwtlicense.html;md5=c5c69645704bbe5b502541dc8b8f25a0 \
file://debian/copyright;md5=7fecb9815c8887be096ca82876491a3b \
file://build/mac/installer-resources/License.html;md5=8b1eb407ff164d61266fda749147081a"

SRC_URI = "git://github.com/performancecopilot/pcp.git;protocol=https"

# Modify these as desired
PV = "3.11.10+git${SRCPV}"
SRCREV = "023872a815c9a914c53d755c0447e411440877be"

S = "${WORKDIR}/git"

# NOTE: the following prog dependencies are unknown, ignoring: which xz lzma hdiutil md5sum mkinstallp dpkg dblatex rpmbuild gzip publican qmake-qt4 qshape xmlto pod2man makepkg seinfo git dlltool gawk makedepend qmake-qt5 pkgmk true gmake dtrace rpm bzip2 gtar xconfirm echo qmake
# NOTE: unable to map the following pkg-config dependencies: libmicrohttpd libsystemd-journal
# (this is based on recipes that have previously been built and packaged)
# NOTE: the following library dependencies are unknown, ignoring: nss pfm regex ibmad sasl2 papi gen ibumad nspr
# (this is based on recipes that have previously been built and packaged)
DEPENDS = "libx11 ncurses cairo openssl bison-native zlib readline systemd avahi flex-native"

# NOTE: if this software is not capable of being built in a separate build directory
# from the source, you should replace autotools with autotools-brokensep in the
# inherit line
inherit perlnative python3native pkgconfig pythonnative autotools

# Specify any options you want to pass to the configure script using EXTRA_OECONF:
EXTRA_OECONF = ""
</pre>
It seemed to do a pretty good job. It figured out some licences, detected an autotools project that included python and perl components, added some package dependencies and highlighted some that it couldn't figure out. I was almost there. How hard could it be? Now let's try and build it.
<pre>
$ devtool build pcp
<snip>
| FATAL ERROR: Cannot perform cross-compilation without a file to source
| configuration information from (config.linux is missing)
<snip>
</pre>
I'm not much of an autotools guy but luckily had '''Joshua Lock''' to call on. It turns out that the project wasn't properly set up for cross-compiling. He also spotted that the project wasn't capable of building into a seprate build folder. Luckily if already had a config file for mingw-w64 so I copied it, tweaked it slightly and renamed to config.linux. I added it to the recipe
SRC_URI += "file://config.linux"
However we have to copy the file to the right place. As with Yocto, there are many way to do this. I chose to do it manually before the compile step as
# ${S} is not set when SRC_URI declarations are typically given, so we can't use the subdir parameter
# Doing the copy after unpack would be more natural but as unpack is a pythno python, appends can't be a simple shell command

Articles/Battles with Performance Co-Pilot

2017-06-16T06:00:07Z

Henry Bruce: Henry Bruce moved page Articles/Battles with Performance Co-Pilot to Articles/Battles with Cockpit

#REDIRECT [[Articles/Battles with Cockpit]]

Articles/Battles with Cockpit

2017-06-16T06:00:07Z

Henry Bruce: Henry Bruce moved page Articles/Battles with Performance Co-Pilot to Articles/Battles with Cockpit

I had the task of creating a recipe for the [https://github.com/performancecopilot/pcp Performance Co-Pilot], a system-level performance monitoring and management tool. What didn't help is that I knew nothing about the project itself. I've been working with Yocto for almost 3 years. I'm getting better, but am still learning, as this experience has shown me. Here is my story.

I started by using devtool's recipe generation feature.
$ devtool add devtool add pcp https://github.com/performancecopilot/pcp.git
It gave me a recipe, let's take a look
<pre>
# Recipe created by recipetool
# This is the basis of a recipe and may need further editing in order to be fully functional.
# (Feel free to remove these comments when editing.)

SUMMARY = "System-level performance monitoring and performance management"
# WARNING: the following LICENSE and LIC_FILES_CHKSUM values are best guesses - it is
# your responsibility to verify that the values are complete and correct.
#
# NOTE: multiple licenses have been detected; they have been separated with &
# in the LICENSE value for now since it is a reasonable assumption that all
# of the licenses apply. If instead there is a choice between the multiple
# licenses then you should change the value to separate the licenses with |
# instead of &. If there is any doubt, check the accompanying documentation
# to determine which situation is applicable.
#
# The following license files were not able to be identified and are
# represented as "Unknown" below, you will need to check them yourself:
# COPYING
# man/html/qwtlicense.html
# debian/copyright
# build/mac/installer-resources/License.html
#
# NOTE: spec file indicates the license may be "GPLv2+ and LGPLv2.1+ and CC-BY"
LICENSE = "Unknown"
LIC_FILES_CHKSUM = "file://COPYING;md5=37ab75b580d5aad4ada04260efa3702f \
file://man/html/qwtlicense.html;md5=c5c69645704bbe5b502541dc8b8f25a0 \
file://debian/copyright;md5=7fecb9815c8887be096ca82876491a3b \
file://build/mac/installer-resources/License.html;md5=8b1eb407ff164d61266fda749147081a"

SRC_URI = "git://github.com/performancecopilot/pcp.git;protocol=https"

# Modify these as desired
PV = "3.11.10+git${SRCPV}"
SRCREV = "023872a815c9a914c53d755c0447e411440877be"

S = "${WORKDIR}/git"

# NOTE: the following prog dependencies are unknown, ignoring: which xz lzma hdiutil md5sum mkinstallp dpkg dblatex rpmbuild gzip publican qmake-qt4 qshape xmlto pod2man makepkg seinfo git dlltool gawk makedepend qmake-qt5 pkgmk true gmake dtrace rpm bzip2 gtar xconfirm echo qmake
# NOTE: unable to map the following pkg-config dependencies: libmicrohttpd libsystemd-journal
# (this is based on recipes that have previously been built and packaged)
# NOTE: the following library dependencies are unknown, ignoring: nss pfm regex ibmad sasl2 papi gen ibumad nspr
# (this is based on recipes that have previously been built and packaged)
DEPENDS = "libx11 ncurses cairo openssl bison-native zlib readline systemd avahi flex-native"

# NOTE: if this software is not capable of being built in a separate build directory
# from the source, you should replace autotools with autotools-brokensep in the
# inherit line
inherit perlnative python3native pkgconfig pythonnative autotools

# Specify any options you want to pass to the configure script using EXTRA_OECONF:
EXTRA_OECONF = ""
</pre>
It seemed to do a pretty good job. It figured out some licences, detected an autotools project that included python and perl components, added some package dependencies and highlighted some that it couldn't figure out. I was almost there. How hard could it be? Now let's try and build it.
<pre>
$ devtool build pcp
<snip>
| FATAL ERROR: Cannot perform cross-compilation without a file to source
| configuration information from (config.linux is missing)
<snip>
</pre>
I'm not much of an autotools guy but luckily had '''Joshua Lock''' to call on. It turns out that the project wasn't properly set up for cross-compiling. He also spotted that the project wasn't capable of building into a seprate build folder. Luckily if already had a config file for mingw-w64 so I copied it, tweaked it slightly and renamed to config.linux. I added it to the recipe
SRC_URI += "file://config.linux"
However we have to copy the file to the right place. As with Yocto, there are many way to do this. I chose to do it manually before the compile step as
# ${S} is not set when SRC_URI declarations are typically given, so we can't use the subdir parameter
# Doing the copy after unpack would be more natural but as unpack is a pythno python, appends can't be a simple shell command