gollum
gollum copied to clipboard
Create a resource pool
There are a lot of resources that could be shared between producers and/or even consumers. E.g. if there are 3 kafka based plugins, we have 3 connections to each broker while one would be sufficient. Same goes for any other resource like files, network sockets, etc.
Implement and design a concept where resources of any kind can be added and retrieved from a central resource registry.
- It must be threadsafe
- Cleanup of used resources must be possible across plugins (e.g. via reference counting)
- We don't need a "general resource" type, as the type of the resource is defined by the plugin.
As for the last point: we could try to create something like a general resource by trying to find a general interface, but this is overkill and will create more problem than it solves. The registry should be treated like a cache, not as a hub to enable writing a generic producer or consumer.