fasten icon indicating copy to clipboard operation
fasten copied to clipboard

CGMerger throws OutOfMemoryException

Open vigna opened this issue 2 years ago • 3 comments

On branch develop, trying to merge with CGMerger the dependencies related to revision fasten://io.7mind.izumi:distage-extension-config_2.12$1.0.6 (id=76535) causes an out-of-memory exception, even using 128GB of RAM.

To replicate, try

echo '+fasten://io.7mind.izumi:distage-extension-config_2.12$1.0.6' | java -Xmx128G eu.fasten.core.search.SearchEngine jdbc:postgresql://monster:5432/fasten_java fastenro /mnt/fasten/data/callable-index/ foo

on monster. Note that the SearchEngine class has no correlation with the problem—it is just an easy way to run CGMerger over the revision above. After about 20 minutes, CGMerger throws the following exception:

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at java.base/java.util.Arrays.copyOf(Arrays.java:3689)
	at java.base/java.util.ArrayList.grow(ArrayList.java:238)
	at java.base/java.util.ArrayList.addAll(ArrayList.java:710)
	at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
	at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
	at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
	at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
	at eu.fasten.core.merge.CGMerger.createUniversalCHA(CGMerger.java:701)
	at eu.fasten.core.merge.CGMerger.<init>(CGMerger.java:222)
	at eu.fasten.core.search.SearchEngine.from(SearchEngine.java:511)
	at eu.fasten.core.search.SearchEngine.fromRevision(SearchEngine.java:479)
	at eu.fasten.core.search.SearchEngine.fromRevision(SearchEngine.java:464)
	at eu.fasten.core.search.SearchEngine.main(SearchEngine.java:736)

Other revisions from the same group id merge without problems.

vigna avatar Mar 18 '22 13:03 vigna

A similar behavior occurs with id=104122.

vigna avatar Mar 19 '22 09:03 vigna

This looks indeed like a bug. @ashkboos is on vacation right now, but will look into this as soon as he is back.

proksch avatar Mar 19 '22 20:03 proksch

Also with 1122038, 132690, 132691, and 573488.

vigna avatar Mar 21 '22 16:03 vigna