Benchmark result (June 6, 2018)

[AkihiroSuda]

Travis log available: https://travis-ci.org/AkihiroSuda/buildbench/builds/388656138

Buildbench version

https://github.com/AkihiroSuda/buildbench/commit/70c88896bacbd23752195e715339e9d2c48b04d0

Env

TravisCI Ubuntu Trusty ( https://docs.travis-ci.com/user/reference/overview/ )

• Ext4
• 7.5GB RAM
• 2 Cores, bursted

Tested tools

• Docker 18.05-ce
• moby/buildkit@376093d234ca9446a3d4dc9956d2eb3afff54baf
• genuinetools/img@1a47aed9
• projectatomic/buildah@f90b6c0
• gcr.io/kaniko-project/executor@sha256:501056bf52f3a96f151ccbeb028715330d5d5aa6647e7572ce6c6c55f91ab374 (Uploaded time: May 18, 2018 at 5:13:31 AM UTC+9)

Benchmark: examples/ex01

• foo-1: clean environment
• foo-2: keep foo-1 cache storage, touch buildContext/build-bench-dummy, and then build the image again
• Tested 5 times

ID	Took (seconds)
docker-1	20,15,18,16,17 (avg: 17.2)
docker-2	1,2,2,1,1 (avg: 1.4)
buildkit-1	12, 13, 8, 9, 8 (avg: 10.0)
buildkit-2	2, 1, 1, 1, 1 (avg: 1.2)
img-1	11,11,10,13,14 (avg: 11.8)
img-2	2,2,2,1,2 (avg: 1.8)
buildah-1	15, 15, 13, 21, 14 (avg: 15.6)
buildah-2	11, 11, 15, 14, 17 (avg: 13.6)
kaniko-1	21, 18, 16, 16, 15 (avg: 17.2)
kaniko-2	15, 18, 18 26, 16 (avg: 18.6)

20180606-ex01.txt

Benchmark: moby/moby@fd2f2a919e392b96de74795ae9af2dc5e510bc4c

• The image could not be built with Buildah and Kaniko, due to lack of support for FROM anotherstage (EDIT: fixed typo s/anotherimage/anotherstage/)
• Tested single time only

ID	Took (seconds)
docker-1	387
docker-2	8
buildkit-1	266
buildkit-2	2
img-1	450
img-2	6

20180606-moby.txt

[AkihiroSuda]

Restarted CI, but still img is slow for moby/moby. Weird...

https://travis-ci.org/AkihiroSuda/buildbench/builds/389694818

$ cat ${CSV}
docker-1,406,1528460306,1528460712
docker-2,6,1528460712,1528460718
buildkit-1,265,1528460727,1528460992
buildkit-2,1,1528460992,1528460993
img-1,416,1528460999,1528461415
img-2,7,1528461415,1528461422