kafka-elasticsearch-injector
kafka-elasticsearch-injector copied to clipboard
SchemaRegistry cache my fail in concurrent miss scenario
I this following code may not work in concurrent miss
scenario
https://github.com/inloco/kafka-elasticsearch-injector/blob/master/src/kafka/encode_decode.go#L115-L142
the solution described here with ChanLocker pattern might fix it. https://lakefs.io/2020/09/23/in-process-caching-in-go-scaling-lakefs-to-100k-requests-second/
Hi @xmlking! Thanks for raising this issue.
The current code does load the schema multiple times in a concurrent miss scenario, but the performance does not suffer that much, as the number of schemas is not large (for most cases) and the loading time is not prohibitive.
The ChanLocker pattern looks like a great idea, though. PRs are welcome!