scylla-cdc-rust
scylla-cdc-rust copied to clipboard
Add support for JSON based consumers
I have some use cases where I would like to send JSON in a CDC consumer.
Right now, the consumer only accepts Cql, and extracting/processing the CqlRow can be a bit complicated, especially if you want to build some JSON response from it.
I think it would be beneficial if you didn't have to do any conversion client-side in this case. Would it be reasonable to add support for accepting JSON rows instead of Cql in the consumer implementation?
I've implemented a draft for what that could look like: https://github.com/scylladb/scylla-cdc-rust/compare/main...rasviitanen:scylla-cdc-rust:rasviitanen/json-consumer?expand=1.
In short:
- Add a trait
ConsumerTypeand implement it forJsonandCql - Add a marker type to the consumer trait and default it to Cql:
Consumer<T = Cql> where T: ConsumerType - Change the
consume_cdcfunction toasync fn consume_cdc(&mut self, data: T::Value<'_>) -> anyhow::Result<()>.T::Valuefor Cql will be the currentCDCRowand forJsonit's basically just a json text string.
If there's any interest from Scylla's side, I'd be happy to submit a PR for it. I'd understand if you wish to implement it some other way, or not have the feature available at all, but this felt decent enough to me!
Thank you!