So you want to run Docker in Concourse? Well this is the guide for you!
Let' clarify what it is we want to do. We want to be able to run
docker-compose inside a task in Concourse to bring up our application along side some other services (i.e. Redis, Postgres, MySQL, etc.).
Thankfully this challenge has been solved by the community! There are a few "Docker-in-Docker" images designed to run in Concourse that are maintained by the community. Here's a short list made from a cursory search, in no particular order:
You can also opt to build your own or fork the above images.
All of the above repositories have their own example pipelines that you can use to get started. What follows are some bits of information that are useful to know when using these task images.
You can verify this by looking at the bash scripts for each of the above images which all take inspiration from the docker-image resource. Read the
sanitize_cgroups function to see what exactly is being mounted from the host. (tldr: mount all cgroups as read-write)
Externalize All Images
You should avoid having Docker fetch any images from inside your task step where you are running
docker-compose. You should externalize these as image resources if they're a dependency of your application (e.g. Postgres, MySQL).
To ensure Docker doesn't try to fetch the images itself you can use
docker load and
docker tag to load your externalized images into Docker. meAmidos's has a great example pipeline that does exactly that.
meAmidos also makes two great points about why you should externalize your image:
- If the image comes from a private repository, it is much easier to let Concourse pull it, and then pass it through to the task.
- When the image is passed to the task, Concourse can often get the image from its cache.
That's all you need to know to run Docker inside Concourse!