grizzly
grizzly copied to clipboard
(Possibly) Incorrect initial max memory size when building a direct memory buffer pool
When creating a buffer pool using PooledMemoryManager
, then DEFAULT_HEAP_USAGE_PERCENTAGE (3%) of the heap is used to calculate the max pool size per slice:
final long heapSize = Runtime.getRuntime().maxMemory();
final long memoryPerSubPool = (long) (heapSize * percentOfHeap / numberOfPools);
https://github.com/eclipse-ee4j/grizzly/blob/master/modules/grizzly/src/main/java/org/glassfish/grizzly/memory/PooledMemoryManager.java#L170C1-L171C89
However, this calculation is not correct when direct buffers are created, as the amount of direct memory is not reported by Runtime.getRuntime().maxMemory()
. Thus, on machines with large heap, but a small direct memory, the initialization fails with OOM. This simple code:
// run with -XX:MaxDirectMemorySize=128m
public static void main(String[] args) {
MemoryManager<? extends Buffer> POOLED_BUFFER_ALLOCATOR =
new PooledMemoryManager(
512 * 1024, // base chunk size
1, // number of pools
2, // grow factor per pool, ignored, see above
4, // expected concurrency
PooledMemoryManager.DEFAULT_HEAP_USAGE_PERCENTAGE,
PooledMemoryManager.DEFAULT_PREALLOCATED_BUFFERS_PERCENTAGE,
true // direct buffers
);
}
fails with:
Exception in thread "main" java.lang.OutOfMemoryError: Cannot reserve 524288 bytes of direct buffer memory (allocated: 133701632, limit: 134217728)
at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:127)
at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360)
at org.glassfish.grizzly.memory.PooledMemoryManager$PoolSlice.allocate(PooledMemoryManager.java:685)
at org.glassfish.grizzly.memory.PooledMemoryManager$PoolSlice.<init>(PooledMemoryManager.java:569)
at org.glassfish.grizzly.memory.PooledMemoryManager$Pool.<init>(PooledMemoryManager.java:432)
at org.glassfish.grizzly.memory.PooledMemoryManager.<init>(PooledMemoryManager.java:175)
at dev.kofemann.playground.GerizzlyAllocateorOOM.main(GerizzlyAllocateorOOM.java:65)
I assume that with max direct memory 128MB, creating a single slice with 4x 512KB chunks should be possible.
I think, for direct buffers instead of heapSize
max direct memory size should be used, something like:
final long maxMem = isDirect ? jdk.internal.misc.VM.maxDirectMemory()
: Runtime.getRuntime().maxMemory();
final long memoryPerSubPool = (long) (maxMem * percentOfHeap / numberOfPools);