So you want to run Docker in Concourse? Well this is the guide for you!
Let’ clarify what it is we want to do. We want to be able to run docker-compose inside a task in Concourse to bring up our application along side some other services (i.e. Redis, Postgres, MySQL, etc.).
Thankfully this challenge has been solved by the community! There are a few “Docker-in-Docker” images designed to run in Concourse that are maintained by the community.
In this blog post we’re going to cover how to use git and Concourse to automatically set, update, and archive your pipelines using the set_pipeline step. No longer will you need to use fly set-pipeline to update any of your pipelines!
For consistency we will refer to the pipeline that contains all the set_pipeline steps as the parent pipeline. The pipelines created by the set_pipeline steps will be called child pipelines.
In this blog post we are going to show how to build and publish container images using the oci-build task and registry-image resource. This post assumes you understand how to build container images with Dockerfile’s and publish to Docker Hub or another image registry using the docker cli.
If you just want to see the pipeline, scroll to the bottom or click here. What follows is a detailed explanation of what each part of the pipeline does.
Understanding how task inputs and outputs work in Concourse can be a little confusing initially. This post will walk you through a few example pipelines to show you how inputs and outputs work within a single Concourse job. By the end you should understand how inputs and outputs work within the context of a single job.
Let’s define some jargon first.
step : A step is a container running code within the context of a Concourse job.