OOMs during heap analysis

[damianw]

Description

We regularly see OOMs from LeakCanary during analysis - we've already enabled the separate process, and skip all initialization in that process, but the issue has persisted. So, seems that the LeakCanary analyzer is using most if not all of the heap in the process for these large heap dumps.

Here's one example stack trace, which is how the issue most frequently manifests:

java.lang.OutOfMemoryError: Failed to allocate a 32 byte allocation with 903480 free bytes and 866KB until OOM, target footprint 201326592, growth limit 201326592; giving up on allocation because <1% of heap free after GC.
	at shark.HprofHeapGraph.wrapIndexedObject(HprofHeapGraph.kt:355)
	at shark.HprofHeapGraph.findObjectByIdOrNull(HprofHeapGraph.kt:161)
	at shark.HprofHeapGraph.findObjectById(HprofHeapGraph.kt:144)
	at shark.HeapObject$HeapInstance.getInstanceClass(HeapObject.kt:353)
	at shark.internal.FieldInstanceReferenceReader.read(FieldInstanceReferenceReader.kt:70)
	at shark.internal.ChainingInstanceReferenceReader.read(ChainingInstanceReferenceReader.kt:24)
	at shark.internal.ChainingInstanceReferenceReader.read(ChainingInstanceReferenceReader.kt:13)
	at shark.internal.DelegatingObjectReferenceReader.read(DelegatingObjectReferenceReader.kt:17)
	at shark.internal.PathFinder.findPathsFromGcRoots(PathFinder.kt:164)
	at shark.internal.PathFinder.findPathsFromGcRoots(PathFinder.kt:135)
	at shark.HeapAnalyzer.findLeaks(HeapAnalyzer.kt:275)
	at shark.HeapAnalyzer.analyzeGraph(HeapAnalyzer.kt:253)
	at shark.HeapAnalyzer.analyze$shark(HeapAnalyzer.kt:217)
	at shark.HeapAnalyzer.analyze(HeapAnalyzer.kt:166)
	at leakcanary.internal.AndroidDebugHeapAnalyzer.analyzeHeap(AndroidDebugHeapAnalyzer.kt:158)
	at leakcanary.internal.AndroidDebugHeapAnalyzer.runAnalysisBlocking(AndroidDebugHeapAnalyzer.kt:59)
	at leakcanary.internal.RemoteHeapAnalyzerWorker$startRemoteWork$1.run(RemoteHeapAnalyzerWorker.kt:22)
	at android.os.Handler.handleCallback(Handler.java:938)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loopOnce(Looper.java:201)
	at android.os.Looper.loop(Looper.java:288)
	at android.os.HandlerThread.run(HandlerThread.java:67)

Build.VERSION.SDK_INT: 31
Build.MANUFACTURER: Google
LeakCanary version: 2.8.1
Analysis duration: 15451 ms
Heap dump file path: /storage/emulated/0/Download/leakcanary-.../2022-04-25_13-56-39_280.hprof
Heap dump timestamp: 1650910304785

Occasionally, we also get crashes like this:

java.lang.RuntimeException: Not enough memory to allocate buffers for rehashing: 2097152 -> 4194304
	at shark.internal.hppc.LongLongScatterMap.allocateBuffers(LongLongScatterMap.kt:326)
	at shark.internal.hppc.LongLongScatterMap.allocateThenInsertThenRehash(LongLongScatterMap.kt:358)
	at shark.internal.hppc.LongLongScatterMap.set(LongLongScatterMap.kt:103)
	at shark.internal.DominatorTree.updateDominated(DominatorTree.kt:51)
	at shark.internal.PathFinder$VisitTracker$Dominated.visited(PathFinder.kt:63)
	at shark.internal.PathFinder.enqueue(PathFinder.kt:224)
	at shark.internal.PathFinder.findPathsFromGcRoots(PathFinder.kt:170)
	at shark.internal.PathFinder.findPathsFromGcRoots(PathFinder.kt:135)
	at shark.HeapAnalyzer.findLeaks(HeapAnalyzer.kt:275)
	at shark.HeapAnalyzer.analyzeGraph(HeapAnalyzer.kt:253)
	at shark.HeapAnalyzer.analyze$shark(HeapAnalyzer.kt:217)
	at shark.HeapAnalyzer.analyze(HeapAnalyzer.kt:166)
	at leakcanary.internal.AndroidDebugHeapAnalyzer.analyzeHeap(AndroidDebugHeapAnalyzer.kt:158)
	at leakcanary.internal.AndroidDebugHeapAnalyzer.runAnalysisBlocking(AndroidDebugHeapAnalyzer.kt:59)
	at leakcanary.internal.RemoteHeapAnalyzerWorker$startRemoteWork$1.run(RemoteHeapAnalyzerWorker.kt:22)
	at android.os.Handler.handleCallback(Handler.java:938)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loopOnce(Looper.java:201)
	at android.os.Looper.loop(Looper.java:288)
	at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by: java.lang.OutOfMemoryError: Failed to allocate a 33554456 byte allocation with 21003496 free bytes and 20MB until OOM, target footprint 201326592, growth limit 201326592
	at shark.internal.hppc.LongLongScatterMap.allocateBuffers(LongLongScatterMap.kt:322)
	... 19 more

Build.VERSION.SDK_INT: 31
Build.MANUFACTURER: Google
LeakCanary version: 2.8.1
Analysis duration: 48021 ms
Heap dump file path: /storage/emulated/0/Download/leakcanary-.../2022-04-19_17-48-22_751.hprof
Heap dump timestamp: 1650901181176

Steps to Reproduce

I am not able to reproduce the crash end-to-end on a standalone app, but a sample HPROF file that produces an OOM can be found here. (This file has been privately shared with @pyricau only at the moment)

Version Information

• LeakCanary version: 2.8.1
• Android OS version: 31
• Gradle version: 7.4