Sbt-assembly is using lots of native memory

[ashangit]

Issue description

On some of our build relying on sbt-assembly to create shaded jar we have observed big usage of native memory. For a 6G max heap JVM, the pod memory usage is above 12G

Analysing the native usage with libtcmalloc_and_profiler.so.4 and google-pprof we observed that the native usage is due to native usage of Inflater used to unzip jar:

Src code ref of the native inflater code: Java_java_util_zip_Inflater_init and Java_java_util_zip_Inflater_inflateBytesBytes

Proposal

Current code keep all jar file handler which leads to all native inflater being kept with all the memory those are using

In order to avoid this, the proposal is to load all entry data from each jar as ByteArrayInputStream instead of keeping the stream and immediately close the jar once all entries are loaded

Here the native memory usage after this change

The counterpart is an increase of the heap usage. On our specific build we had to increase heap size to 7G but pod memory usage has decreased below 10G: