sonic-buildimage
sonic-buildimage copied to clipboard
sonic-net
02.Version cache - docker cache build framework
During docker build, host files can be passed to the docker build through docker context files. But there is no straightforward way to transfer the files from docker build to host.
This feature provides a tricky way to pass the cache contents from docker build to host. It tar's the cached content and encodes them as base64 format and passes it through a log file with a special tag as 'VCSTART and VCENT'.
Slave.mk in the host, it extracts the cache contents from the log and stores them in the cache folder. Cache contents are encoded as base64 format for easy passing.
Why I did it
How I did it
How to verify it
Which release branch to backport (provide reason below if selected)
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
- [ ] 202205
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)
![azure-pipelines[bot] avatar](https://avatars.githubusercontent.com/in/9426?v=4 "Published by a pub.dev verified publisher"){kind=small}
@xumia @liushilongbuaa submitter has taken the time to split the original code PR to smaller submissions for review, pls help take a look
Makefile.cache provides the same feature.
Yes, it is an extension of the DPKG caching framework.
- I don't know what you want to copy.
- From pipeline result, folder target/vcache is only 3KB and cache.tgz is empty.
It is just a framework to copy files from the docker builder to the host. Currently, In this patch, nothing has been added to this file. This PR has only the framework part of the version cache framework.
The following patches in this series will add files to it.
https://github.com/sonic-net/sonic-buildimage/pull/12003 https://github.com/sonic-net/sonic-buildimage/pull/12004 https://github.com/sonic-net/sonic-buildimage/pull/12005
and more coming on.
Did you try https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs?
Did you try https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs?
tried the below commands, but it doesn't generate the expected out file in the host.
#cat Dockerfile FROM debian:buster AS build-stage RUN mkdir -p /test RUN echo "MY CNT" > /test/myfile.txt
FROM scratch AS export-stage COPY --from=build-stage /test /
#docker build --no-cache -o out . #ls Dockerfile
shilongliu@lsl-vm2:~/test$ ls
Dockerfile
shilongliu@lsl-vm2:~/test$ cat Dockerfile
FROM debian:buster
RUN echo HHH > /bin/a.log
From scratch
COPY --from=0 /bin/a.log a.log
shilongliu@lsl-vm2:~/test$ DOCKER_BUILDKIT=1 docker build --output out .
[+] Building 0.1s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/debian:buster 0.0s
=> [stage-0 1/2] FROM docker.io/library/debian:buster 0.0s
=> CACHED [stage-0 2/2] RUN echo HHH > /bin/a.log 0.0s
=> CACHED [stage-1 1/1] COPY --from=0 /bin/a.log a.log 0.0s
=> exporting to client 0.0s
=> => copying files 32B 0.0s
shilongliu@lsl-vm2:~/test$ find .
.
./Dockerfile
./out
./out/a.log
Using SCP/RCP to copy the files from the docker builder to the host might not work in all the combinations. This creates a security and authentication problem in the multiuser environment even if we use public authentication keys. The sonic-slave docker image is built directly on the build server which requires SCP access on the build server to copy the files.
The -o option is not working with the image tag option. If we run with the -o option, it only generates the output and it does not generate the docker image.
# DOCKER_BUILDKIT=1 docker build --no-cache -t test:latest -o out .
=> exporting to client
=> copying files 56B
# find out out out/a.log # docker images |grep "test *latest" < --- NO test:latest IMAGE IS GENERATED --- >
When we run without the -o option, the docker image is generated, but the output is empty.
# DOCKER_BUILDKIT=1 docker build --no-cache -t test:latest .
=> exporting to image
=> => exporting layers
=> => writing image sha256:f1201b5415c4d7ad943f55f32e1a8102d9467da2c0f791ca822ce38c791f6a39
=> => naming to docker.io/library/test:latest
# find out < --- NO OUTPUT GENERATED --- > # docker images |grep "test *latest" test latest f1201b5415c4 12 seconds ago 28B
The below sequence solves both the docker image generation and intermediate cache file output to the host. But the only problem is that we need to run the docker build command three times. Though we are reusing the image layers, still little overhead on the build time, and requires a lot of changes in the docker build process in the sonic. Let me know your thought on this?
cat Dockerfile
# Base docker build
FROM debian:buster as build
RUN dd if=/dev/zero of=/root/cache.data bs=100M count=1
# Copy the cache data to host
From scratch as output
COPY --from=sonic-image-tmp:latest /root/cache.data cache.data
# Clean up the cache data
FROM sonic-image-tmp:latest as final
RUN rm /root/cache.data
cat build.sh
rm -rf out; docker rmi sonic-image:latest
DOCKER_BUILDKIT=1 docker build --no-cache --tag sonic-image-tmp:latest --target build .;
DOCKER_BUILDKIT=1 docker build --cache-from sonic-image-tmp:latest --target output -o out .;
DOCKER_BUILDKIT=1 docker build --no-cache --cache-from sonic-image-tmp:latest --target final --tag sonic-image:latest .
docker rmi sonic-image-tmp:latest
find out; docker images |grep "sonic-image *latest"
======================================================
bash -x ./build.sh
-
rm -rf out
-
docker rmi sonic-image:latest Untagged: sonic-image:latest Deleted: sha256:7799604f2b5b5649aed23df18fdb20f9125f1d7d45279c583b1dee993379a322
-
DOCKER_BUILDKIT=1
-
docker build --no-cache --tag sonic-image-tmp:latest --target build . => CACHED [build 1/2] FROM docker.io/library/debian:buster => [build 2/2] RUN dd if=/dev/zero of=/root/cache.data bs=100M count=1 => => exporting layers => => writing image sha256:db76b329801e567580cc45e294709855c443f6acab629cc89d804eea8867f99f => => naming to docker.io/library/sonic-image-tmp:latest
-
DOCKER_BUILDKIT=1
-
docker build --cache-from sonic-image-tmp:latest --target output -o out . => importing cache manifest from sonic-image-tmp:latest => FROM docker.io/library/sonic-image-tmp:latest => => resolve docker.io/library/sonic-image-tmp:latest => CACHED [output 1/1] COPY --from=sonic-image-tmp:latest /root/cache.data cache.data => exporting to client
=> => copying files 104.88MB -
DOCKER_BUILDKIT=1
-
docker build --no-cache --cache-from sonic-image-tmp:latest --target final --tag sonic-image:latest . => [internal] load metadata for docker.io/library/sonic-image-tmp:latest => importing cache manifest from sonic-image-tmp:latest => CACHED [final 1/2] FROM docker.io/library/sonic-image-tmp:latest => [final 2/2] RUN rm /root/cache.data => exporting to image => => exporting layers => => writing image sha256:ad3df1f88f3af5353bebd157744a1b3eab294fead83e58497b6c57787011f8cc => => naming to docker.io/library/sonic-image:latest
-
docker rmi sonic-image-tmp:latest Untagged: sonic-image-tmp:latest Deleted: sha256:db76b329801e567580cc45e294709855c443f6acab629cc89d804eea8867f99f
===============OUTPUT=============
-
find out out out/cache.data
-
docker images | grep 'sonic-image *latest' sonic-image latest ad3df1f88f3a 1 second ago 219MB
@Kalimuthu-Velappan , the solution is good.
Can we only use two docker build steps? Split the Dockerfile into two parts, Dockerfile1 and Dockerfile2, Dockerfile1 is to do the normal installation without cleanup, Dockerfile2 is only to do the cleanup, it can be generated from a common j2 template for all docker builds. We can add a step to copy the content from the image. It is not necessary to depend on DOCKER_BUILDKIT, and makes the copy steps more controllable not in the Dockerfile.
But I think the solution you provided is good enough.
