docker-coala-base
docker-coala-base copied to clipboard
coala
Docker image size is too large
The coala/base docker image is pretty large for what it is. The size is 2.63 GB, that's larger than ubuntu and opensuse combined.
There are two ways to solve this:
- Remove downloaded source code
- Use another distro (preferably Alpine Linux)
So, the size should have be been reduced by clearing the cache 8d9f9ff687015 . It would have been good to have a separate PR for that, with builds before and after that commit so we could see the improvement.
Are there any other changes in the recent changes which might have helped reduce the size?
https://cloud.docker.com/app/coala/repository/docker/coala/base/tags says pre (rebuilt after all this) is still 1Gb, which is what it reports for 0.8 and 0.9 which do not have all the recent attempts at optimising the image size.
Other than using a smaller distro, probably merge some RUN instructions may help increase the amount of layers but that would kill the ease of debugging / diagnostics.
@jayvdb It's published: https://codein.withgoogle.com/dashboard/task-instances/6600150384377856/
https://github.com/coala/docker-coala-base/pull/66 and https://github.com/coala/docker-coala-base/pull/67 removed a few hundred Mb from the image. An image rebuild would be useful.
An analysis build of files > 5Mb
Before any commands are run (including before packages are added):
$ find / -xdev -type f -size +10M
/usr/share/icu/57.1/icudt57l.dat
At the end:
$ find / -xdev -type f -size +10M
/usr/bin/node
/usr/bin/shellcheck
/usr/bin/hlint
/usr/share/go/contrib/src/golang.org/x/tools/.git/objects/pack/pack-e7faa2d81b805c54fc75d515c067e0e5ca57fa82.pack
/usr/share/icu/57.1/icudt57l.dat
/usr/lib/python2.7/site-packages/pyxb/bundles/common/raw/xhtml1.wxs
/usr/lib64/go/pkg/linux_amd64_dynlink/libstd.so
/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/amd64/server/libjvm.so
/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/amd64/server/classes.jsa
/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/rt.jar
/usr/lib64/libopenblas_openmp.so.0
/usr/lib64/julia/sys.so
/usr/lib64/libruby2.2-static.a
/usr/lib64/gcc/x86_64-suse-linux/6/lto1
/usr/lib64/gcc/x86_64-suse-linux/6/libasan.a
/usr/lib64/gcc/x86_64-suse-linux/6/libstdc++.a
/usr/lib64/gcc/x86_64-suse-linux/6/libgfortran.a
/usr/lib64/gcc/x86_64-suse-linux/6/cc1
/usr/lib64/gcc/x86_64-suse-linux/6/cc1plus
/usr/lib64/gcc/x86_64-suse-linux/6/f951
/usr/lib64/libjulia.so.0.5.0
/root/.RLibrary/stringi/libs/icudt55l.dat
/root/.RLibrary/stringi/libs/stringi.so
/root/.RLibrary/igraph/libs/igraph.so
/root/pmd-bin-5.4.1/lib/scala-compiler-2.10.4.jar
/root/nltk_data/tokenizers/punkt.zip
/root/.julia/v0.5/METADATA/.git/objects/pack/pack-ae7cd04fc8509df91bd5b36ee8d9104e3bb8bf4a.pack
/root/.bundle/cache/compact_index/rubygems.org.443.29b0360b937aa4d161703e6160654e47/versions
/var/lib/rpm/Packages
/coala/.git/objects/pack/pack-708ef23f4cf60fb7b7436b1a3031701e5648b015.pack
Some of those files are not necessary.
I've added another task for this, which is simpler and can be used to find smaller size reductions, which are still useful.
An example of that task is removing R-core-doc
R-recommended-packages seems to be unnecessary. https://github.com/rhemon/docker-coala-base/commit/ff853d71c0e36db87e5efb5be2a1fae214480bd6 https://travis-ci.org/rhemon/docker-coala-base/builds/189255334
@rhemon it still pulls R-recommended-packages because it's a dependency for one of the R packages
libasan3 seems to be unnecessary (although it only reduces size by 0.001 GB) https://travis-ci.org/rhemon/docker-coala-base/builds/189275655 https://github.com/rhemon/docker-coala-base/commit/b75ae0376db9b242af4630b27a1177badfe7f35b
Found these unnecessary packages and erased them in PR https://github.com/coala/docker-coala-base/pull/80 :
libasan3
python3-six
python-cssselect
python-curses
python-javapackages
python-lxml
python-Pygments
python-pyxb
python-setuptools
python-six
python-xml
texlive-gsftopk
texlive-kpathsea
xorg-x11-fonts
xorg-x11-fonts-core
Found that postfix was unnecessary and also thanks for @jayvdb for mentioning R-core-doc
The travis build: https://travis-ci.org/GnarledHarpy06/docker-coala-base/builds/191619054
btw https://github.com/coala/docker-coala-base/issues/84#issuecomment-272386925 was another set of 9 unnecessary packages, trimming 3 MB from the image.
Remove more unnecessary packages here https://github.com/coala/docker-coala-base/pull/86
aaa_base
dbus-1
dbus-1-x11
gio-branding-openSUSE
kmod
libthai-data
lksctp-tools
logrotate
openssh
openslp
perl-Net-DBus
postfix
R-core-doc
sysconfig-netconfig
systemd-presets-branding-openSUSE
wicked-service
https://github.com/rancher/dapper is a tool to wrap any existing build tool in an consistent environment. It could reduce the size of images significantly but I'm not sure if it's suitable for coala.
I will have a try.
As an experiment, I tried to make the installation of lua-devel and luarocks temporary.
The result is the image increases by 33Mb from 2.208 GB to 2.241 GB. See commit and build. This is counter intuitive, and suggests there are extra steps needed to completely tidy up after using zypper.
Update: jayvdb/docker-coala-base@d628dd8 build is slightly better, but with the same result.
This post is a WIP cataloging some a bunch of my tiny un-PR'd stuff
- dart-no-doc - 1 Mb reduction
On the other hand
-use-phpsniffer-package - adds 5mb
Due to voodoo behind the scenes in openSUSE, the image size is down to 2.143 GB (after https://github.com/coala/docker-coala-base/pull/158 which doesnt affect size)
After https://github.com/coala/docker-coala-base/pull/185 , we're back up to 2.222 GB
EDIT: And after https://github.com/coala/docker-coala-base/pull/188 , 2.23 GB
After https://github.com/coala/docker-coala-base/pull/241 , the size is back down to 2.2Gb