How To Build and Publish a Container Image

June 19, 2020

In this blog post we are going to show how to build and publish container images using the oci-build task and registry-image resource. This post assumes you understand how to build container images with Dockerfile’s and publish to Docker Hub or another image registry using the docker cli.

If you just want to see the pipeline, scroll to the bottom or click here. What follows is a detailed explanation of what each part of the pipeline does.

First we need a Dockerfile. You can store this in your own repo or reference the github.com/concourse/examples repo. The rest of this post assumes you use the examples repo. All files in this blog post can be found in the examples repo.

The Dockerfile

We are going to use a very basic Dockerfile so we can focus on building the Concourse pipeline.

FROM busybox

RUN echo "I'm simple!"
COPY ./stranger /stranger
RUN cat /stranger

Defining Pipeline Resources

Now we can start building out our pipeline. Let’s declare our resources first. We will need one resource to pull in the repo where our Dockerfile is located, and a second resource pointing to where we want to push the built container image to.

There are some variables in this file that we will fill out later.

resources:
# The repo with our Dockerfile
- name: concourse-examples
  type: git
  icon: github
  source:
    uri: https://github.com/concourse/examples.git
    branch: master

# Where we will push the image to
- name: simple-image
  type: registry-image
  icon: docker
  source:
    repository: ((image-repo-name))/simple-image
    username: ((registry-username))
    password: ((registry-password))

Create a Job

Next we will create a job that will build and push our container image.

jobs:
- name: build-and-push

Retrieve the Dockerfile

The first step in the job plan will be to retrieve the repo where our Dockerfile is.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples

Build the Container Image

The second step in our job will build the container image.

To build the container image we are going to use the oci-build-task. The oci-build-task is a container image that is meant to be used in a Concourse task to build other container images. Check out the README in the repo for more details on how to configure and use the oci-build-task in more complex build scenarios.

Let’s add a task to our job plan and give it a name.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image

All configuration of the oci-build-task is done through a task config. Viewing the README from the repo we can see that the task needs to be run as a privileged task on a linux worker.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux

To use the oci-build-task container image we specify the image_resource that the task should use.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task

Next we will add concourse-examples as an input to the build task to ensure the artifact from the get step (where our Dockerfile is fetched) is mounted in our build-task-image step.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task
      inputs:
      - name: concourse-examples

The oci-build-task outputs the built container image in a directory called image. Let’s add image as an output artifact of our task so we can publish it in a later step.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task
      inputs:
      - name: concourse-examples
      outputs:
      - name: image

Next we need to tell the oci-build-task what the build context of our Dockerfile is. The README goes over a few other methods of creating your build context. We are going to use the simplest use-case. By specifying CONTEXT the oci-build-task assumes a Dockerfile and its build context are in the same directory.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task
      inputs:
      - name: concourse-examples
      outputs:
      - name: image
      params:
        CONTEXT: concourse-examples/Dockerfiles/simple

The last step is specifying what our build-task-image should execute. The oci-build-task container image has a binary named build located in its PATH in the /usr/bin directory. We’ll tell our task to execute that binary, which will build our container image.

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task
      inputs:
      - name: concourse-examples
      outputs:
      - name: image
      run:
        path: build
      params:
        CONTEXT: concourse-examples/Dockerfiles/simple

At this point in our job the container image is built! The oci-build-task has saved the container image as a tarball named image.tar in the image artifact specified in the task outputs. This tar file is the same output you would get if you built the container image using Docker and then did docker save.

Publish the Container Image

Now let’s push the container image to an image registry! For this example we’re pushing to Docker Hub using the registry-image resource. You can use the registry-image resource to push to any image registry, private or public. Check out the README.md for more details on using the resource.

To push the container image add a put step to our job plan and tell the regstry-image resource where the tarball of the container image is.

The put step will push the container image using the information defined in the resource’s source, when we defined the pipeline’s resources.

This is where you’ll need to replace the three variables found under resource_types. You can define them statically using fly’s --var flag when setting the pipeline. (In production make sure to use a credential management system to store your secrets!)

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task
      inputs:
      - name: concourse-examples
      outputs:
      - name: image
      params:
        CONTEXT: concourse-examples/Dockerfiles/simple
      run:
        path: build
  - put: simple-image
    params:
      image: image/image.tar

The Entire Pipeline

Putting all the pieces together, here is our pipeline that builds and pushes (publishes) a container image.

resources:
# The repo with our Dockerfile
- name: concourse-examples
  type: git
  icon: github
  source:
    uri: https://github.com/concourse/examples.git
    branch: master

# Where we will push the image
- name: simple-image
  type: registry-image
  icon: docker
  source:
    repository: ((image-repo-name))/simple-image
    username: ((registry-username))
    password: ((registry-password))

jobs:
- name: build-and-push
  plan:
  - get: concourse-examples
  - task: build-task-image
    privileged: true
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: vito/oci-build-task
      inputs:
      - name: concourse-examples
      outputs:
      - name: image
      params:
        CONTEXT: concourse-examples/Dockerfiles/simple
      run:
        path: build
  - put: simple-image
    params:
      image: image/image.tar

You can set the pipeline with the following fly command, updating the variable values with real values the pipeline can use. The behaviour is similar to docker push:

fly -t <target> set-pipeline -p build-and-push-image \
    -c ./examples/pipelines/build-and-push-simple-image.yml \
    --var image-repo-name=<repo-name> \
    --var registry-username=<user> \
    --var registry-password=<password>