[FEATURE REQ] Cache count configuration in Eventhubs SDK

[chuckz1321]

Is your feature request related to a problem? Please describe. As tested, Java Eventhubs SDK may cache specific count of events and will relased the memory when the count reach a threshold. Correct me if I misunderstand. Because I find a parameter in C# SDK.

Also, here's my test. The main function just have a processor and it will just print some info to console. I use Jconsole to monitor the memory. When message body is "hello", and I set a small prefetch count.

When the message body is larger, the memory used increases. Customer may start multiple threads to consume messages in one Java process. And I assume if the message body is large like 0.5MB or larger, it will cause OOM based on the limited heap size configuration.

Describe the solution you'd like So could we please add the controle to the cache size? If the memory raise is not caused by my consumption, please correct it and glad to hear your insights.

Feel free to ping me on Teams. [email protected]

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [ ] Description Added
• [ ] Expected solution specified

[chuckz1321]

Hi team, any insights on this issue?

[alzimmermsft]

@conniey, @anuchandy are there any updates for this?

[conniey]

Hey,

As tested, Java Eventhubs SDK may cache specific count of events and will released the memory when the count reach a threshold. Correct me if I misunderstand.

Based on your configuration, the Event Hub client library will cache the prefetch count events, so 500 events. It fetches more events when a portion of the events have been consumed by the caller and fills the prefetch queue back to the prefetch count.

We stop referencing the underlying AMQP message when it is successfully decoded. We don't control objects that are garbage collected.

There are many factors that can increase memory usage. The prefetch number and message size can affect it.

Before limiting your heap size, I would run your application for some time with a message of average size and see the max heap size trend when it stabilises.

[chuckz1321]

I am confused about the cache count and the value of prefetch parameter. You can compare the first and second pic. They received same messages(I mean the event body). The only difference is the prefetch count, first one is 500 and 20 for second one. And their peek memory values are all 100MB+

I noticed you said the reference of the event will be released when the processor complete the messages. But from this point, I highly suspect some variables still refer to it. :(

[chuckz1321]

Do you need the code? I can share it. Just the batch received demo from github

[chuckz1321]

@conniey Any update?

[anuchandy]

The prefetch configures the events to be fetched from the broker and buffered. The batchSize controls draining of this buffer. As the draining happens, the library communicates with the broker and tries to keep the buffer full.

When the event size increases, it is expected to see a positive delta in memory usage. Multiple factors are involved, e.g., The Heap for the buffer increases because of the increase in event size stored in each buffer slot. The operation of the reading and decoding event body must allocate a byte [] array depending on the event size. (Other factors internal allocation in JVM IO APIs, the allocation from low-level AMQP library, etc.)

---

If the goal is to identify how much memory to reserve for the application - measure and limit Heap accordingly. To expand on what @conniey shared earlier, a common way to measure is -

Run the application until it has reached a steady state. E.g., the application and JVM have loaded anything it caches [e.g., domain objects, class types, static instances, object pools], has created a maximum number of connections, and has a peak expected load). Now force a full GC using tools such as JVisualVM / JConsole. Check how much memory is occupied after the full GC. You want to size the Heap such that only 30% is occupied after the full GC; use this value to set the max heap size (-Xmx). Size each container memory to have an "additional ~1 GB" memory for the "non-heap" need for the JVM instance.

In a nutshell, measure the application for a long duration with messages of expected average size and see the max heap size trend when it stabilizes, then only limit Java Heap based on the observed value. You want to run your application against an Event Hub with enough messages to keep the SDK busy during the measuring.

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!