Dependencies cache in docker multi-stage builds

During migration to Docker multi stage builds noticed that downloading of dependencies takes a quite long time which breaks work flow and delays build feedback.

Example of building Kubernetes Java client in Docker image with Java 16:

FROM openjdk:16.0.1-jdk as builder
RUN mkdir /k8s-client
ADD . /k8s-client/
WORKDIR /k8s-client
RUN ./mvnw package
FROM openjdk:16.0.1
COPY --from=builder /k8s-client/kubernetes/target/*.jar .

Building image:

DOCKER_BUILDKIT=1 docker build . -t k8s-java:1

Execution times:

=> [internal] load build definition from Dockerfile                      0.0s
 => => transferring dockerfile: 269B                                      0.0s
 => [internal] load .dockerignore                                         0.0s
 => => transferring context: 2B                                           0.0s
 => [internal] load metadata for docker.io/library/openjdk:16.0.1         1.5s
 => [internal] load metadata for docker.io/library/openjdk:16.0.1-jdk     1.5s
 => [builder 1/5] FROM docker.io/library/openjdk:16.0.1-jdk@sha256        0.0s
 => [internal] load build context                                        12.0s
 => => transferring context: 447.73MB                                    11.9s
 => CACHED [stage-1 1/2] FROM docker.io/library/openjdk:16.0.1@sha256     0.0s
 => CACHED [builder 2/5] RUN mkdir /k8s-client                            0.0s
 => [builder 3/5] ADD . /k8s-client/                                      1.9s
 => [builder 4/5] WORKDIR /k8s-client                                     0.0s
 => [builder 5/5] RUN ./mvnw package                                    138.0s

To optimize download time, found it quite useful to create of base image for “builder” container with all dependencies, run build commands inside which may trigger downloading of “deltas” if a new dependency was added or transient dependency got updated.

Run builder container and mount inside application repo:

docker run -it -v $(pwd):/tmp/app --entrypoint /bin/bash openjdk:16.0.1-jdk

Download dependencies inside of container:

# cd /tmp/app
# ./mvnw dependency:resolve

Identify hostname and exit from the container

# cat /etc/hostname
c2372446377e
# exit

Persist downloaded changes in the container as a new image with dedicated tag:

docker commit c2372446377e denis256/k8s-deps:1

Now, tagged image can be used as a builder image, and during builds will be re-used cached dependencies:

FROM denis256/k8s-deps:1 as builder
RUN mkdir /k8s-client
ADD . /k8s-client/
WORKDIR /k8s-client
RUN ./mvnw package

FROM openjdk:16.0.1
COPY --from=builder /k8s-client/kubernetes/target/*.jar .

Execution times:

=> [internal] load build definition from Dockerfile                                 0.0s
 => => transferring dockerfile: 270B                                                0.0s
 => [internal] load .dockerignore                                                   0.0s
 => => transferring context: 2B                                                     0.0s
 => [internal] load metadata for docker.io/library/openjdk:16.0.1                   3.2s
 => [internal] load metadata for docker.io/denis256/k8s-deps:1                      3.0s
 => [auth] denis256/k8s-deps:pull token for registry-1.docker.io                    0.0s
 => [auth] library/openjdk:pull token for registry-1.docker.io                      0.0s
 => CACHED [stage-1 1/2] FROM docker.io/library/openjdk:16.0.1@sha256:              0.0s
 => [internal] load build context                                                   0.5s
 => => transferring context: 2.82MB                                                 0.4s
 => [builder 1/5] FROM docker.io/denis256/k8s-deps:1@sha256:                        0.1s
 => => resolve docker.io/denis256/k8s-deps:1@sha256:                                0.0s
 => [builder 2/5] RUN mkdir /k8s-client                                             0.2s
 => [builder 3/5] ADD . /k8s-client/                                                4.2s
 => [builder 4/5] WORKDIR /k8s-client                                               0.0s
 => [builder 5/5] RUN ./mvnw package                                                10.1s

Reduction of package time from 138.0s to 10.1s (last line)

Future work:

• E2E automation of process and re-run each week in a secured environment to avoid downloading of unexpected dependencies;
• Automated report building with all dependencies
• Scanning of dependencies for vulnerabilities
• Extend approach on other languages like GoLang