Add API to replace rd_kafka's default allocator

[sarkanyi]

trafficstars

Add API to replace rd_kafka's default allocator with a custom implementation

[sarkanyi]

I think I covered all requests. Could you please re-check and merge if ok? Also please check and merge: https://github.com/edenhill/librdkafka/pull/2405 because it's related. I can rebase both if needed

[edenhill]

Will re-review soon, but this will not be merged until after the next release.

[edenhill]

With your PR merged to add mem_malloc() et.al. I'm guessing this PR is no longer needed, an application can use LD_PRELOAD.. to replace the rd_malloc()* calls, right?

[sarkanyi]

With your PR merged to add mem_malloc() et.al. I'm guessing this PR is no longer needed, an application can use LD_PRELOAD.. to replace the rd_malloc()* calls, right?

To be honest it would be preferable to have an API, either this - I'll change to have rd_str_dup & co also added, or function hooks something like: "typeof (&rd_malloc) malloc_fn = rd_malloc_hook ? rd_malloc_hook : malloc; void *p = malloc_fn(sz);" -set via a setter function?

[sarkanyi]

Most big libraries use some sort of API to set the custom allocator, ie: jansson has void json_set_alloc_funcs(json_malloc_t malloc_fn, json_free_t free_fn), it makes it more explicit. LD_PRELOAD is quite invasive, even if we'd overload rd_funcs

[edenhill]

Okay, a single function to set the allocators once on program startup before calling any other librdkafka function (in particular before global_init is called) should be fine and avoid the need for locking.

[sarkanyi]

Hi, is this still on track for 1.8? Do you need me to do anything else on it?

[sarkanyi]

@edenhill won't this change make it into 1.9.0? Thanks

[cla-assistant[bot]]

All committers have signed the CLA.