prometheus-cpp icon indicating copy to clipboard operation
prometheus-cpp copied to clipboard

Avoid temporary objects

Open gjasny opened this issue 2 years ago • 2 comments

When collecting many metrics, the intermediate std::vector<MetricFamily> objects can heavily grow in size and cause memory fragmentation (see #646). This PR implements direct text-serialization into a buffer array whose buffers are 1 MByte at maximum.

The extreme memory fragmentation that I saw with the glibc allocator as well as the huge memory spikes during collection are now gone. There are now multiple other optimization possibilities, but this is the bare minimum needed.

gjasny avatar Dec 28 '23 20:12 gjasny

@owent, @esigo, @lalitb: FYI: this PR would change the Collectable signature.

Did you ever run into problems with excessive number of metrics? It's nice to see that prometheus-cpp can cope with 2 million counters. But do such setups exist in the wild?

gjasny avatar Dec 28 '23 21:12 gjasny

@owent, @esigo, @lalitb: FYI: this PR would change the Collectable signature.

Did you ever run into problems with excessive number of metrics? It's nice to see that prometheus-cpp can cope with 2 million counters. But do such setups exist in the wild?

Is there any way to tell otel-cpp whether to use the new class Serializer to get Serializer? I found no macro to check the ABI changes.

owent avatar Dec 29 '23 11:12 owent