JuliaLinearAlgebra
Reduce size of artifacts
I have an app that uses MKL. I want to put it into a docker container, and it shall be as small as possible. I see the following artifacts:
ufechner@ufryzen:/tmp/testdepot/artifacts$ du -h 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/
12K 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/share/licenses/MKL
16K 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/share/licenses
20K 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/share
525M 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/lib
525M 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/
ufechner@ufryzen:/tmp/testdepot/artifacts/03b04466a9b7dc64deb348ce55333edc5dd6b0ce/lib$ ls
libmkl_avx2.so libmkl_cdft_core.so.2 libmkl_intel_lp64.so libmkl_sequential.so.2
libmkl_avx2.so.2 libmkl_core.so libmkl_intel_lp64.so.2 libmkl_tbb_thread.so
libmkl_avx512.so libmkl_core.so.2 libmkl_intel_thread.so libmkl_tbb_thread.so.2
libmkl_avx512.so.2 libmkl_def.so libmkl_intel_thread.so.2 libmkl_vml_avx2.so
libmkl_blacs_intelmpi_ilp64.so libmkl_def.so.2 libmkl_mc3.so libmkl_vml_avx2.so.2
libmkl_blacs_intelmpi_ilp64.so.2 libmkl_gf_ilp64.so libmkl_mc3.so.2 libmkl_vml_avx512.so
libmkl_blacs_intelmpi_lp64.so libmkl_gf_ilp64.so.2 libmkl_rt.so libmkl_vml_avx512.so.2
libmkl_blacs_intelmpi_lp64.so.2 libmkl_gf_lp64.so libmkl_rt.so.2 libmkl_vml_cmpt.so
libmkl_blacs_openmpi_ilp64.so libmkl_gf_lp64.so.2 libmkl_scalapack_ilp64.so libmkl_vml_cmpt.so.2
libmkl_blacs_openmpi_ilp64.so.2 libmkl_gnu_thread.so libmkl_scalapack_ilp64.so.2 libmkl_vml_def.so
libmkl_blacs_openmpi_lp64.so libmkl_gnu_thread.so.2 libmkl_scalapack_lp64.so libmkl_vml_def.so.2
libmkl_blacs_openmpi_lp64.so.2 libmkl_intel_ilp64.so libmkl_scalapack_lp64.so.2 libmkl_vml_mc3.so
libmkl_cdft_core.so libmkl_intel_ilp64.so.2 libmkl_sequential.so libmkl_vml_mc3.so.2
Can I delete the .so files that I do not need? How can I find out which one I am using?
To answer my second question:
sudo grep /tmp/testdepot/* /proc/*/maps | grep mkl
shows me which .so files of mkl are in use.
To answer my first question: The following script deletes all but the two .so files that I need:
cd /tmp/testdepot/artifacts/03b04466a9b7dc64deb348ce55333edc5dd6b0ce/lib/
mkdir bak
mv libmkl_rt.so.2 bak
mv libmkl_rt.so bak
mv libmkl_core.so.2 bak
mv libmkl_core.so bak
find . -maxdepth 1 -type f,l -exec rm -f {} \;
mv bak/* .
rmdir bak
cd /tmp/testdepot
The size of the MKL artifacts is reduced from 525M to 83M.
c5dd6b0ce/
12K 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/share/licenses/MKL
16K 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/share/licenses
20K 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/share
83M 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/lib
83M 03b04466a9b7dc64deb348ce55333edc5dd6b0ce/
Each of those libraries provide different parts of the MKL functionality, and if you delete the libraries you can cause MKL to then not be portable across machines, or have the various optimized code paths for other architectures - which I think may make it crash because it is expecting them to be there.
I know. But if I need a small docker image to be used within a small company usually only one or two architectures need to be supported. So a documented way to strip down the size to the architectures that are required would be nice.
