Say you want to use an Azure DevOps pipeline to build your containerized app. This scenario is of course supported and you would leverage all the built-in stuff (built-in container image, default container tasks, hosted build agent, etc.). You can even choose between the UI or YAML to configure the build pipeline.
But what if you cannot work with the default build container image? What if you need additional tools installed to succesfully build your own app? Well, one option is to provide your own build image. The way it works, is that you can configure an Azure DevOps pipeline to fetch and use a custom image from a Docker Repo (such as Docker Hub) or an Azure Container Registry. Our steps:
- Create our own build image, add our required tools
- Build, tag and upload this image to an Azure Container Registry
- Create a YAML build definition (the UI won’t do us much good here)
- Have the build fetch our container image
- Run the YAML build using this image (this will also build, tag and push our own app image)
The first part depends of course on your own requirements. In my case I use a pretty standard OpenJDK image and only add the Scala and the Scala Built Tools (sbt).
FROM openjdk:8-slim ENV SCALA_VERSION 2.11.12 ENV SBT_VERSION 0.13.16 RUN \ apt-get update && \ apt-get install -y libltdl7 && \ apt-get install -y sudo && \ apt-get install -y curl # Install Scala RUN \ curl -fsL https://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C root/ && \ echo >> /root/.bashrc && \ echo "export PATH=~/scala-$SCALA_VERSION/bin:$PATH" >> /root/.bashrc # Install sbt RUN \ curl -L -o sbt-$SBT_VERSION.deb https://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \ dpkg -i sbt-$SBT_VERSION.deb && \ rm sbt-$SBT_VERSION.deb && \ apt-get install sbt && \ sbt sbtVersion
Note: if you do not add libltdl7 you might receive an error such as: “/usr/bin/docker: error while loading shared libraries: libltdl.so.7”
Note: we also add sudo, this is not installed by default and we need it to make our custom image work with the Azure DevOps Container Job. See the hacky part further down.
We can now build our image, tag our image and push it to an Azure Container Registry.
docker build . -t pipeline-build-image docker tag pipeline-build-image demoxyz.azurecr.io/pipeline-build-image:latest docker push demoxyz.azurecr.io/pipeline-build-image
After this, we continue with creating our build definition using a YAML file. You can create one by adding a a simple file to your repository. The default name is “azure-pipelines.yml” and just enter a name for our build pipeline. For now this is enough, so commit the file.
Next step is to configure our pipeline. Add one and select the YAML template:
We need to select the Hosted Ubuntu Agent Pool and select the path to our YAML definition:
If we now edit our new build pipeline, we can continue creating our pipeline. We need a couple of things to make it all work.
resources: containers: - container: build_container image: demoxyz.azurecr.io/pipeline-build-image:latest endpoint: AzureCR options: '-v /usr/bin/docker:/usr/bin/docker -v /var/run/docker.sock:/var/run/docker.sock'
This part tells configures a custom container for the build. It requires an image (url to our Azure Container Registry + image and tag). It will require a Service Connection of the type Dockerregistry, so you will need to set this up separately.
The “options” play an important role. They will bind mount two volumes:
- /usr/bin/docker for the docker binaries. We need these since we want to output a container as part of our build process.
- /var/run/docker.sock. This is the socket the Docker daemon listens on. Our docker build step will connect to this, because we don’t want to run a container daemon inside a container instance :)
container: build_container variables: azureSubscriptionEndpoint: AzureRM azureContainerRegistry: demoxyz.azurecr.io steps: - script: | set -ex sudo groupadd -o -g $(stat --format='%g' /var/run/docker.sock) docker sudo usermod -a -G docker $(whoami) displayName: Allow current user to access the docker socket
This instructs the build to actually use the image and sets up another service connection (of the Azure Resource Manager). This time, the connection is used by the Azure DevOps Container Jobs.
This last bit is the hacky part. Because we run our tasks as a normal user, we get a permission denied when we access the socket. To avoid this we add the current user (whoami) to a group that has access.
And finally we can add the default YAML container jobs. They rely on the variables mentioned earlier and are well documented here:
- task: Docker@1 displayName: Container registry login inputs: command: login azureSubscriptionEndpoint: $(azureSubscriptionEndpoint) azureContainerRegistry: $(azureContainerRegistry) - task: Docker@1 displayName: Build image inputs: command: build azureSubscriptionEndpoint: $(azureSubscriptionEndpoint) azureContainerRegistry: $(azureContainerRegistry) dockerFile: Dockerfile imageName: $(Build.Repository.Name) - task: Docker@1 displayName: Tag image inputs: command: tag azureSubscriptionEndpoint: $(azureSubscriptionEndpoint) azureContainerRegistry: $(azureContainerRegistry) imageName: $(azureContainerRegistry)/$(Build.Repository.Name):latest arguments: $(azureContainerRegistry)/$(Build.Repository.Name):$(Build.BuildId) - task: Docker@1 displayName: Push image inputs: command: push azureSubscriptionEndpoint: $(azureSubscriptionEndpoint) azureContainerRegistry: $(azureContainerRegistry) imageName: $(Build.Repository.Name):$(Build.BuildId)
Success, we can now “docker build” our app using a custom build image!
The code used in this article is available on github: https://github.com/yuriburger/custom-build-container-image