jitsu
jitsu copied to clipboard
Support for alternative events queue broker
Problem of current design:
- Events routing in Jitsu is hardcoded.
- Also many components are hardwired. E.g. http handlers and destinations
- Only 2 implementation of events queue supported: Redis and Inmemory. Enterprise client may what to use other solutions like Kafka
Proposals:
- use events broker to detach component from each other
- use abstraction to allow supporting multiple event streaming solution: kafka, redpanda, pulsar,...
- don't reinvent routing mechanism for solution with built-in one
- keep supporting inmemory and redis with handmade routing mechanism
- (separate task) Expand usage of events broker for more cases: user recognition?, events cache?, sync statues, post handle destinations and notification, errors reporting, metrics reporting ( e.g. much more convenient to consolidate the most all metrics logic in one service, other service need to simply send status event )