AutoBuilder Cluster Setup

From Yocto Project
Jump to navigationJump to search

While the instructions below are targeted at setting up a new AutoBuilder cluster, they are equally valid for an single machine with combined controller and worker, except you probably won't use a NAS for storage. The controller portion requires only 2 GB of local disk space; the worker requires 2+ TB to hold all the build temp files and git repos. If build artifacts and a local sstate mirror are included, additional worker space is required.

As root

  • Install base OS with minimal target
  • Install the requirements from http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html
    • (Note: if you plan to build the poky-eclipse-neon plugin, openjdk-headless 1.8 is required.)
  • If you plan to run the sanity tests (which is typical for an AB), the following additional packages need to be installed:
    • GitPython for Python2 and Python3
    • xsltproc
    • gnupg
    • vncserver (whichever one is provided by the distro is fine)
  • Add pokybuild user (uid=6000, or for older clusters 1006)
  • Add pokybuild user to "users" group (gid 100) (if "users" group does not exist, create it)
  • Following the instructions from https://wiki.yoctoproject.org/wiki/How_to_enable_KVM_for_Poky_qemu, give /dev/kvm permissions to the pokybuild user.
  • If a "kvm" group does not exist, add it to /etc/group and ensure /dev/kvm gid is set to "kvm" and is mode 660. Add pokybuild user to the "kvm" group.
  • git clone --single-branch https://git.yoctoproject.org/git/poky /root/poky (in /root for historical reasons, should change in the future)
  • If adding tap devs for sanity testing. Install https://downloads.yoctoproject.org/releases/yocto/yocto-{latest_release}/toolchain/x86_64/poky-glibc-x86_64-core-image-sato-core2-64-toolchain-{relver}.sh to /opt/poky.
  • Add startup scripts to create tap devs and start vncserver, in /etc/rc.local, /etc/init.d/boot.local, or as systemd unit files, which should perform the following:
    • cd /root/poky/scripts/; ./runqemu-gen-tapdevs 6000 100 32 /opt/poky/sysroots/x86_64-pokysdk-linux/
    • /bin/su - pokybuild -c "/usr/bin/vncserver"
    • Note that the location of the buildtools might actually be different than the example shown here, depending on the default path, etc. If you run into a tunctl error when generating the tapdevs, it's likely a path problem.
    • You will need to run vncserver once as the pokybuild user, and set the password. The password of "builder" is generally used.
  • Disable or edit firewall rules to allow all connections to the tap devices. This allows sanity tests to connect to booted images.
  • Install nfs-utils or nfs-common, add Domain to /etc/idmapd.conf, fstab entry, mount NAS under /srv/autobuilder.
  • We use more watches then the defaults generally allow so, echo "fs.inotify.max_user_watches=65536" >> /etc/sysctl.conf

As pokybuild

  • Set up buildset-config, either:
    • Add ssh key to clone production yocto-autobuilder
    • git clone https://git.yoctoproject.org/git/yocto-autobuilder && cd yocto-autobuilder && git remote add prod git@git.yoctoproject.org:yocto-autobuilder-production && git fetch prod && git checkout -b yoctoio prod/yoctoio
  • or:
    • edit one of the several examples included in repo e.g. cp -r buildset-config.controller buildset-config
  • update config/autobuilder.conf and yocto-worker/buildbot.tac
  • Start the worker
    • touch ~/yocto-autobuilder/.setupdone
    • . yocto-autobuilder-setup
    • ./yocto-start-autobuilder worker
Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=AutoBuilder_Cluster_Setup&oldid=29065"