The current buildsystem

The current (and working) buildsystem consists of the following parts:

Figure 5. Buildsystem structure

Buildsystem structure

The physical host

The physical host is a box running a vserver-capable kernel. System is a usual Red Hat Linux distribution plus some extra-packages (vserver related). It does not need much disk-space; 500 MB should be enough for minimal installations. On this host the 'djinnid' daemon must be running with the commandset defined by the package.

The buildmaster vserver

This is a vserver, where the buildmaster person(s) are starting the builds and are sending the built packages to a separate signing server. Requirements on this vserver are minimal also.

The build will be initiated by start-build <dist> <package-url> <md5sum> <gpg-key#> which is starting a buildslave in a separate context within an unbreakable chroot. This operations will require privileges to mount tmpfs filesystems or to create device inodes. The ``new_bslave'' command of djinni will be used for this task. In detail, this action

  • mounts the readonly buildslave with ``--bind''
  • creates /dev in the slave as a tmpfs and populates it; dev-inodes on a readonly volume are not working, e.g. ``echo >/dev/null'' would fail there.
  • creates /var in the slave as a tmpfs
  • mounts a slave-specific, writable buildroot with ``--bind'' at /var/lib/mach

There exists a ``stop_bslave'' command with revokes this actions but does not do any removals/cleanups.

The buildslave environment

This environemt is created as a usual vserver and can be used in this way. But for build-server tasks it is just an ordinary chroot environment (no init step) which uses the unbreakable chroot feature of vserver kernels. This environment MUST be mounted readonly or write-protected in other ways.

The current buildsystem solves this by having a readonly mounted loopdevice at /mnt/bslave in the host. Now, each ``new_bslave'' djinni command will execute mount --bind /mnt/bslave "." in the root-directory of the buildslave.

The requirements of the buildslave are small; 200 MB should be enough. A buildslave does not build more than one package; at its startup all old data will be wiped and the new buildroot be initialized. The package-build itself is done with a slightly modified mach.

Since buildroot creation requires privileges to mount a /proc filesystem and to create special inodes, djinni will be called by mach with the ``prepare_machroot'' and ``shutdown_machroot'' commands.

The buildroot(s)

Each buildslave needs a separate buildroot where the real build happens. This buildroot is a directory in a workspace partition and is mounted with ``--bind'' at /var/lib/mach. The workspace partition contains temporary data only and should have a very fast filesystem; ext2 would be ideally for it. It needs much space; 2GB are required e.g. for a glibc build. Since each slave will need this, slave-count * 2GiB should be reserved for this area.

The buildroot-area contains mach files (cache, apt-configuration, ...) and will be wiped for each build. It can be initialized with prebuild filesystem images to speedup the build.