feat: Customizable disposal policy wrt concurrent sinks limit

[julealgon]

The current implementation only allows users to control cleanup by setting the maximum amount of concurrent sinks in the map.

In my scenario, I'd like to have more control over this behavior, perhaps by implementing policies like:

• Dispose the sink after a certain amount of time after it's created
• Dispose the sink after a certain amount of time after being inactive (no events received)
• Dispose the sink after a certain log event is received (arbitrary predicate on a given event)

Perhaps it would be possible to expose an extension point by introducing an abstraction, like a SinkPool, then allow consumers to implement their own versions with custom creation and disposal semantics.

Having this feature would allow me to have better control with my unbounded sinks. Today, I can't leave this option null because it would create a memory leak in the application, but controlling the number of open sinks doesn't match our business needs.

[nblumhardt]

Thanks for the heads-up. I can definitely see the limitations in the default policy.

It would be cool to see a sketch of something like SinkPool, especially if it meant the sink could ship with some more implementations (the current one, LIFO, FIFO, time-since-write, etc.) I think, before we shipped the abstraction, we'd need some of these examples to prove that the abstraction fits enough use cases, anyway.

There's quite a bit of work in getting to something we could review further, which I won't have time to attempt personally, but marking this "up-for-grabs" in case someone's keen to explore it.

Cheers!

[nblumhardt]

Closing this one as stale, though contributions in this area are still welcome if anyone's keen.