- You have a Dockerfile for when you want to build your prod image
- You have another Dockerfile for when you want to run the unit tests
- Those two images do not depend upon one another
- This is wrong
The Problem With The Situation
Build time. BUILD. TIME. Now if you want to run the tests, you have to gather all of your dependencies, which is fine, because you want the dependencies to be there when you run the tests.
But what happens when you want to build the thing to deploy it, say your tests pass? You gotta wait for those dependencies to be gathered AGAIN. This is long. I am of the Internet Generation, I AM EASILY BORED.
Well. I mean. “A solution”. “My solution”. I humbly propose to you the following: Multi-stage builds.
Look, it’s gonna hurt less, in the end, I promise.
Let’s get this show on the road:
FROM golang:1.11.1-alpine3.8 AS base ENV CGO_ENABLED 0 # Install tools required for project # Run `docker build --no-cache .` to update dependencies RUN apk update && apk add --no-cache git make protobuf-dev RUN go get github.com/golang/dep/cmd/dep FROM base AS deps # Now you go get all the project's dependencies. # When using modules, you can simplify all this. COPY . $GOPATH/src/path/to/your-project WORKDIR $GOPATH/src/path/to/your-project RUN dep ensure -vendor-only -v ## Run some tests FROM deps AS tests RUN go vet ./... RUN go test ## Build our service FROM deps AS build WORKDIR $GOPATH/src/path/to/your-project RUN go build -o /service FROM scratch AS out COPY --from=build /service / CMD ["service"]
Separating it in three build stages is important as you’ll see.
What, that’s it?
Hang on, little tomato. You still need to build those.
Say you want to run the tests first, and then only build and push the final image if those go through?
docker build --target tests . && docker build --target out --tag
my-service:v1.0.0 && docker push my-service:v1.0.0
The above is quite basic, but you can go from there and build it
out however you want. Doing so will use the Docker cache locally,
which means the second
docker call will actually build the image
out starting from the
deps intermediary image. This can drastically
improve your CI build time.
A Note About CI
In some cases, it might be wise to distribute builds across a cluster of build machines. Were you to do that, I’d recommend tagging intermediary images and pushing them to a private repository; the obvious caveat is that intermediary images can become quite fat, especially if you have build-time dependencies added in the container. For instance, one of my base images easily became 350+ mb. It might be feasible to push out the image from your CI server to your registry, network-wise, but it’s still a nonzero cost, so probably take that into account.
My preferred approach is to try and benefit from the local Docker cache and GNU Parallel so that the jobs that use the same base image (
deps in our example) are built concurrently, and make Parallel exit with a non-zero status code if any of the commands fail by running parallel with
Hopefully reading this was at least a fraction useful to you as it was for me writing it.