AutoBuilder Cluster Setup: Difference between revisions

From Yocto Project
Jump to navigationJump to search
m (Bill Randle moved page AutoBuilderClusterSetup to AutoBuilder Cluster Setup: make title more readable)
mNo edit summary
Line 13: Line 13:
* Following the instructions from https://wiki.yoctoproject.org/wiki/How_to_enable_KVM_for_Poky_qemu, give /dev/kvm permissions to the pokybuild user.
* Following the instructions from https://wiki.yoctoproject.org/wiki/How_to_enable_KVM_for_Poky_qemu, give /dev/kvm permissions to the pokybuild user.
* If a "kvm" group does not exist, add it to /etc/group and ensure /dev/kvm gid is set to "kvm" and is mode 660. Add pokybuild user to the "kvm" group.
* If a "kvm" group does not exist, add it to /etc/group and ensure /dev/kvm gid is set to "kvm" and is mode 660. Add pokybuild user to the "kvm" group.
* git clone --single-branch http://git.yoctoproject.org/git/poky /root/poky  (in /root for historical reasons, should change in the future)
* git clone --single-branch https://git.yoctoproject.org/git/poky /root/poky  (in /root for historical reasons, should change in the future)
* Install http://downloads.yoctoproject.org/releases/yocto/yocto-{latest_release}/toolchain/x86_64/poky-glibc-x86_64-core-image-sato-core2-64-toolchain-{relver}.sh to /opt/poky.
* Install https://downloads.yoctoproject.org/releases/yocto/yocto-{latest_release}/toolchain/x86_64/poky-glibc-x86_64-core-image-sato-core2-64-toolchain-{relver}.sh to /opt/poky.
* Add startup scripts to create tap devs and start vncserver, in /etc/rc.local, /etc/init.d/boot.local, or as systemd unit files, which should perform the following:
* Add startup scripts to create tap devs and start vncserver, in /etc/rc.local, /etc/init.d/boot.local, or as systemd unit files, which should perform the following:
** cd /root/poky/scripts/; ./runqemu-gen-tapdevs 6000 100 32 /opt/poky/sysroots/x86_64-pokysdk-linux/
** cd /root/poky/scripts/; ./runqemu-gen-tapdevs 6000 100 32 /opt/poky/sysroots/x86_64-pokysdk-linux/
Line 25: Line 25:
* Set up buildset-config, either:
* Set up buildset-config, either:
** Add ssh key to clone production yocto-autobuilder
** Add ssh key to clone production yocto-autobuilder
** git clone http://git.yoctoproject.org/git/yocto-autobuilder && cd yocto-autobuilder && git remote add prod git@git.yoctoproject.org:yocto-autobuilder-production && git fetch prod && git checkout -b yoctoio prod/yoctoio
** git clone https://git.yoctoproject.org/git/yocto-autobuilder && cd yocto-autobuilder && git remote add prod git@git.yoctoproject.org:yocto-autobuilder-production && git fetch prod && git checkout -b yoctoio prod/yoctoio
* or:
* or:
** edit one of the several examples included in repo e.g. cp -r buildset-config.controller buildset-config
** edit one of the several examples included in repo e.g. cp -r buildset-config.controller buildset-config

Revision as of 14:41, 27 August 2016

While the instructions below are targeted at setting up a new AutoBuilder cluster, they are equally valid for an single machine with combined controller and worker, except you probably won't use a NAS for storage. The controller portion requires only 2 GB of local disk space; the worker requires 2+ TB to hold all the build temp files and git repos. If build artifacts and a local sstate mirror are included, additional worker space is required.

As root

  • Install base OS with minimal target
  • Install the requirements from http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html
    • (Note: if you plan to build the poky-eclipse-neon plugin, openjdk-headless 1.8 is required.)
  • If you plan to run the sanity tests (which is typical for an AB), the following additional packages need to be installed:
    • GitPython for Python2 and Python3
    • xsltproc
    • vncserver (whichever one is provided by the distro is fine)
  • Add pokybuild user (uid=6000, or for older clusters 1006)
  • Add pokybuild user to "users" group (gid 100) (if "users" group does not exist, create it)
  • Following the instructions from https://wiki.yoctoproject.org/wiki/How_to_enable_KVM_for_Poky_qemu, give /dev/kvm permissions to the pokybuild user.
  • If a "kvm" group does not exist, add it to /etc/group and ensure /dev/kvm gid is set to "kvm" and is mode 660. Add pokybuild user to the "kvm" group.
  • git clone --single-branch https://git.yoctoproject.org/git/poky /root/poky (in /root for historical reasons, should change in the future)
  • Install https://downloads.yoctoproject.org/releases/yocto/yocto-{latest_release}/toolchain/x86_64/poky-glibc-x86_64-core-image-sato-core2-64-toolchain-{relver}.sh to /opt/poky.
  • Add startup scripts to create tap devs and start vncserver, in /etc/rc.local, /etc/init.d/boot.local, or as systemd unit files, which should perform the following:
    • cd /root/poky/scripts/; ./runqemu-gen-tapdevs 6000 100 32 /opt/poky/sysroots/x86_64-pokysdk-linux/
    • /bin/su - pokybuild -c "/usr/bin/vncserver"
  • Disable or edit firewall rules to allow all connections to the tap devices. This allows sanity tests to connect to booted images.
  • Install nfs-utils or nfs-common, add Domain to /etc/idmapd.conf, fstab entry, mount NAS under /srv/autobuilder.

As pokybuild

  • Set up buildset-config, either:
    • Add ssh key to clone production yocto-autobuilder
    • git clone https://git.yoctoproject.org/git/yocto-autobuilder && cd yocto-autobuilder && git remote add prod git@git.yoctoproject.org:yocto-autobuilder-production && git fetch prod && git checkout -b yoctoio prod/yoctoio
  • or:
    • edit one of the several examples included in repo e.g. cp -r buildset-config.controller buildset-config
  • update config/autobuilder.conf and yocto-worker/buildbot.tac
  • Start the worker
    • touch ~/yocto-autobuilder/.setupdone
    • . yocto-autobuilder-setup
    • ./yocto-start-autobuilder worker
Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=AutoBuilder_Cluster_Setup&oldid=20087"