TipsAndTricks/CropsCLIContainers

CROPS or How to Run bitbake/devtool/eSDK/Toaster on Your MacWinux Host

Introduction

CROPS provides CLI containers that allow you to run the set of tools listed in the title on Mac OSX, Windows or Linux. The containers themselves are available on Dockerhub at https://hub.docker.com/u/crops/dashboard/

You may wonder why you would use these containers on Linux. The main reason is it gives you a guaranteed clean environment from which to build. Also it means you don't need to install the dependent packages required by the build tools. Measurements show no speed degradation when running in the container.

How to Get Started

The setup on Mac and Windows requires some extra steps before you get up and running. This is due to the fact that neither the Mac nor the Windows file systems support all the features we need to do a build. To deal with this issue on these OS's we put the build into a persistent docker volume (https://docs.docker.com/engine/tutorials/dockervolumes/) and expose it to the host OS using a Samba container. The detailed instructions for doing this are located at https://github.com/crops/docker-win-mac-docs/wiki

If you plan to use these containers on Linux, you can skip the docker volume and the Samba container instructions as they are unnecessary. The detailed instructions for running the various containers are located in the README.md for each container source on github. Pointers to these instructions for the Poky,eSDK, and Toaster containers are located at the bottom of the above referenced wiki.
NOTE: ALL the CROPS containers take a --help argument in case you forget the options! For example:

 docker run -it --rm crops/poky --help

usage: poky-entry.py [-h] [--workdir WORKDIR] [--id ID]
optional arguments:
 -h, --help         show this help message and exit
 --workdir WORKDIR  The active directory once the container is running. In
                    the abscence of the "id" argument, the uid and gid of the
                    workdir will also be used for the user in the container.
 --id ID            uid and gid to use for the user inside the container. It
                    should be in the form uid:gid

Variations Explained

Poky

The default Poky container is quite generic and will work with most recent releases of YP. Despite the name, it simply contains all the dependent packages required by Poky but not the meta-data. It will work equally well for an OE Core setup. The default container is based on Ubuntu-14.04. We are in the process of creating ones based on the rest of the standard supported Linux Distro's. For the common usage; however, which Distro is inside is irrelevant since everything is already set up for the user.

eSDK

The eSDK container is also generic in that it takes an extensible sdk install script as an argument so it can be used with a fairly arbitrary ext-sdk.

Toaster

Toaster is more complicated. To facilitate quick startup, the Toaster containers include a database prepopulated from the layer index https://layers.openembedded.org. Because of this, the toaster containers come in a couple of varieties.

• Toaster - https://hub.docker.com/r/crops/toaster/ - This is the last official release YP has available. At the time of writing this T&T, for instance, this is krogoth and will soon be morty.
• Toaster-krogoth - https://hub.docker.com/r/crops/toaster-krogoth - The krogoth release.
• Toaster -morty - https://hub.docker.com/r/crops/toaster-morty/ - The RC release. We created this once the morty branch was named/made.
• Toaster-master - https://hub.docker.com/r/crops/toaster-master/ - This is the ever changing master version. A new image is built for every commit to http://git.yoctoproject.org/cgit/cgit.cgi/poky/ on the master branch. We keep ~ 2 weeks back. A docker pull toaster-master will give you the latest commit. If you want a particular commit, look at the tags page: https://hub.docker.com/r/crops/toaster-master/tags/ and you see a number of options.
  • latest - the last successful (passed the Travis test viewable at https://travis-ci.org/crops/toaster-container )
  • latest-<date>-<commitish> - the latest tag points to the last of these. We also keep old ones around for ~ 2 weeks.
  • toaster-next-<date>-<commitish> - This is built using the toaster-next branch of poky-contrib http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/?h=toaster-next. This is intended for people actively developing toaster source code itself.

Operating System Oddities

While the purpose of the CROPS cli containers is to insulate the user from the vagaries of the host os, sometimes the oddness creeps in. Here's some of the host specific issues we've run into so far in case they affect you too.

Mac OSX

Samba/Files

Sometimes you may find yourself unable to delete files from you docker volume from the host os. If you see something like:

$ rm  /Volumes/workdir/.git/objects/pack/pack-67ce6b1222786a14656905bd779637f00fb7225e.pack
override rwxrwxrwx  bavery/staff arch,uchg for/Volumes/workdir/.git/objects/pack/pack-67ce6b1222786a14656905bd779637f00fb7225e.pack? y

it means that a flag has been set that is preventing you from deleting the file. To see what flags are set on the file do:

$ls -lO /Volumes/workdir/.git/objects/pack/pack-67ce6b1222786a14656905bd779637f00fb7225e.pack
-rwxrwxrwx  1 bavery  staff  arch,uchg 34188262 Oct 31 16:07 /Volumes/workdir/.git/objects/pack/pack-67ce6b1222786a14656905bd779637f00fb7225e.pack

In the above example , the uchg (user immutable) and arch (arvhived) flags are set. These need to be cleared using the command chflags To clear them do:

$chflags  noarch,nouchg   /Volumes/workdir/.git/objects/pack/pack-67ce6b1222786a14656905bd779637f00fb7225e.pack

Then you can rm the file. Alternatively, you could do the rm from with in a container and the linux side is less persnickety.

Windows

The biggest issue for Windows so far is that Windows 10 Pro, Enterprise, and Education can use [Docker for Windows https://docs.docker.com/docker-for-windows/] while earlier versions of Windows need to use [docker Toolbox https://docs.docker.com/toolbox/overview/]. Docker for Windows matches what is used on the Mac and is very similar to the experience on Linux. Docker Toolbox insets a VM (by default Virtualbox) in between the host and the cotainers.