Buildbot in Docker

Docker Boulder meetup 1/29/2014

Created by Matt Boersma / matt@sprout.org

Why Bother?

  • Our buildbot is working!
  • Would be some effort to recreate it
  • Python config files in GitHub, at least
  • CI is crucial for OpDemand
  • Docker Hack Day

boot2docker

Docker builds and runs on Mac OS X, but as a client only. Set $DOCKER_HOST in your environment to use a (Linux) docker server transparently.

boot2docker is a solution that runs Docker inside a small VM.

  • Fast and simple (modulo Docker's own bugs)
  • NOTE: Read "hard disk persistence" in README.md

Trusted Builds

  • Connect your GitHub account to index.docker.io
  • Add a public repository with a Dockerfile
  • A post-commit hook will trigger Docker image builds

Its page at the Docker Index indicates a Trusted Build with a ✓ icon, links to the GitHub repo, and the Dockerfile itself.

Project Structure

  • Base buildbot and slave are separate trusted builds
  • OpDemand buildbot and slave pull those images
  • Master shell script starts it all up

Dockerfiles

Docker can act as a builder and read instructions from a text Dockerfile to automate the steps you would otherwise take manually to create an image. Executing docker build will run your steps and commit them along the way, giving you a final image.

Dockerfile: buildbot


# deis/buildbot

FROM stackbrew/ubuntu:12.04
MAINTAINER Matt Boersma <matt@opdemand.com>

ENV HOME /home/buildbot

# install buildbot dependencies from the Ubuntu .deb repository
RUN DEBIAN_FRONTEND=noninteractive apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y \
    python-dev python-pip

# install buildbot from the python package index
RUN pip install buildbot==0.8.8

# run as non-root user "buildbot"
RUN useradd -m -d $HOME buildbot
USER buildbot
RUN cd $HOME && buildbot create-master --relocatable .
ADD master.cfg $HOME/

WORKDIR /home/buildbot
ENTRYPOINT ["/usr/local/bin/twistd", "--nodaemon", "--python", "buildbot.tac"]

# expose the public HTTP site and the twisted PB API interface port
EXPOSE 8010 9989

Dockerfile: buildbot-slave


# deis/buildbot-slave

FROM stackbrew/ubuntu:12.04
MAINTAINER Matt Boersma <matt@opdemand.com>

ENV HOME /home/buildslave

# install buildbot-slave dependencies from the Ubuntu .deb repository
RUN DEBIAN_FRONTEND=noninteractive apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y \
    git-core python-dev python-pip

# install buildbot-slave from the python package index
RUN pip install buildbot-slave==0.8.8

# run as non-root user "buildslave"
RUN useradd -m -d $HOME buildslave
USER buildslave
WORKDIR /home/buildslave

# use an entrypoint script to read Docker link environment variables at runtime
ADD ./entrypoint /entrypoint
ENTRYPOINT ["/entrypoint"]

Repo: opdemand-buildbot

Dockerfile: opdemand-buildbot


# opdemand-buildbot

FROM deis/buildbot
MAINTAINER Matt Boersma <matt@opdemand.com>

# install requirements for OpDemand customizations
USER root
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y python-openssl

# apply OpDemand customizations
USER buildbot
ADD buildutils.py $HOME/
ADD ircbot.py $HOME/
ADD master.cfg $HOME/

Dockerfile: opdemand-buildbot-slave


# opdemand-buildbot-slave

FROM deis/buildbot-slave
MAINTAINER Matt Boersma <matt@opdemand.com>

# OpDemand customizations
USER root
RUN apt-get install -y libpq-dev make python-pip python-virtualenv

# Go back to non-root user
USER buildslave
ADD local_settings.py $HOME/

Script: start_buildbot.sh


#!/bin/bash

# Cleanup
# docker kill buildbot
# docker rm buildbot
# docker kill buildslave1
# docker rm buldslave1
# docker kill buildslave2
# docker rm buildslave2

# DOCKER='docker -H tcp://127.0.0.1:4243/'
DOCKER='docker'

# start the buildbot master
$DOCKER build -rm -t opdemand/buildbot .
$DOCKER run -d -name buildbot -p :8010:8010 -t opdemand/buildbot

# start two buildslaves
pushd slave
$DOCKER build -rm -t opdemand/buildbot-slave .
$DOCKER run -d \
  -name buildslave1 \
  -e BUILDSLAVE_USER=buildslave1 \
  -link buildbot:buildbot \
  -t opdemand/buildbot-slave
$DOCKER run -d \
  -name buildslave2 \
  -e BUILDSLAVE_USER=buildslave2 \
  -link buildbot:buildbot \
  -t opdemand/buildbot-slave
popd

Next: Per-Provider Slaves

Need at least one slave per supported cloud provider to run tests against a live server. (And clean up after itself!)

  • Amazon EC2
  • Rackspace
  • DigitalOcean
  • Vagrant

Next: Docker-in-Docker

Our product Deis is going through a similar metamorphosis: everything is a container. So our git checkout commands in buildbot's master.cfg will need to be changed to docker pull commands. But buildbot is already running inside Docker!

More Info