Errors Thrown in the callbacks are being caught in the library code
When we throw an error within the callback function, should we not allow the user code to catch the the exception rather than catching it in the library code itself?
We shouldn't allow exceptions to propagate into librdkafka's stack. That is written in C and as a consequence isn't aware of what an exception is so throwing something into their stack could lead to corruption of internal structures, memory leaks, etc. Basically your callbacks should not throw. I think the current approach works fine but if you have other suggestions we can look into them.
I was basically writing code where I wanted to stop the execution of a function when there is a message delivery error. Generally I would throw an exception and catch it for doing this. I am not able to think of any other way to do this (maybe modifying from static value from within the callback function). Am I missing some obvious way to do this?
I think you'd have to rely on setting some flag and react to it from outside the callback. It's not ideal at all, but I really wouldn't want an exception to land into C code. That could really mess things up.
Another option would be to pass your own opaque user data into the message builder (as a void*) which you can then retrieve from the message inside the delivery report callback. That user data can be some struct which is global or message specific and you can set state there, signal a condition variable, etc.