lager
lager copied to clipboard
erlang-lager
Motivation to implement overload protection shaper on the handler level instead of sink level?
Hello there. Thank you for continuing to develop lager. Quick question about new flush queue feature. What the reason from architecture point of view to implement this feature in each handler instead of implemente it once on the sink level? Because of this decision each handler plugin developer will have to reimplement this logic again and it's feels like a lot of work and a huge peaces of very similar code. Thank your for reply.
Because different handlers should have the flexibility to decide what constitutes overload and since many handlers can be installed per-sink, it's the only reasonable way to delegate the decision making to handlers.
Can I also ask here why it was decided that queue flush should work only when hwm is defined for the handler? I'm talking about this clause: https://github.com/erlang-lager/lager/blob/06f676aad8ed823b7838a70288cd4d13b5a25ede/src/lager_util.erl#L503
That seems like it might be a situation we need to handle more effectively.
@mrallen1 Thank you for reply. I got your point. But according to my experience its not working so gracefully in real life. Few main issues here:
- Because, overload protection usually mean "drop messages from process queue" and all handlers share same message queue, they will affect each other behavior.
- Most third party handlers doesn't support overload protection. In my use case I have 3 of them, with different combinations of enabled for different environments. So instead of implementing overload protection for each, I just decide to implement "lager overload handler" which actually act exactly like sink level overload protector.
- I can't imagine any use case when such handler level flexibility is needed, can you please share some examples from your experience? When I initially attempted implement overload protection in each handler, implementations were near to copy paste the same code.