fluvio
fluvio copied to clipboard
infinyon
NATS Protocol
Your project is intriguing!
I was looking through your source, and was wondering if you've considered using NATS as a protocol? NATS is lightweight and has cloud native features that would trickle up to making your cli's and operations easier. For a specific example, your users wouldn't need to provision a topic and setup partitions with the cli before sending. You'd also be able to securely provide multi-tenancy from the get-go, which would make it easier to use fluvio in a PaaS/SaaS model.
From an adoption standpoint, you'd be able to reach both the NATS and Kafka ecosystems with the NATS Kafka bridge - best of both worlds. Just a thought....
Full disclosure, I'm a NATS maintainer so have bias, but noticed your project and just wanted to make you aware of NATS.
I wish you luck - we'll be following you guys!
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi @AbstractiveNord, thanks for the question:
We have a NATS connector recently built by the community - https://github.com/infinyon/nats-connector
Does this help? Or is there something specific that you are looking for?
Hi @AbstractiveNord, thanks for the question:
We have a NATS connector recently built by the community - https://github.com/infinyon/nats-connector
Does this help? Or is there something specific that you are looking for?
Hi. Currently, NATS connector has a fatal flaw, it's use NATS Core , but not NATS Jetstream, persistence layer on top of NATS Core. In case of Kafka replacement, that's critical. Jetstream layer provides durable message storage, consumers logic, acknowledges.
Hi @AbstractiveNord, thanks for the question: We have a NATS connector recently built by the community - https://github.com/infinyon/nats-connector Does this help? Or is there something specific that you are looking for?
Hi. Currently, NATS connector has a fatal flaw, it's use NATS Core , but not NATS Jetstream, persistence layer on top of NATS Core. In case of Kafka replacement, that's critical. Jetstream layer provides durable message storage, consumers logic, acknowledges.
Yes we know about Jetstream. Fluvio topics persists messages, has delivery guarantees, ordering, offsets, states as well. So I am trying to understand the use case for Jetstream integration that you are referencing.
Can you share more about the use case where we need to use the persistent model of Kafka, Nats, and Fluvio at the same time?
Hi @AbstractiveNord, thanks for the question: We have a NATS connector recently built by the community - https://github.com/infinyon/nats-connector Does this help? Or is there something specific that you are looking for?
Hi. Currently, NATS connector has a fatal flaw, it's use NATS Core , but not NATS Jetstream, persistence layer on top of NATS Core. In case of Kafka replacement, that's critical. Jetstream layer provides durable message storage, consumers logic, acknowledges.
Yes we know about Jetstream. Fluvio topics persists messages, has delivery guarantees, ordering, offsets, states as well. So I am trying to understand the use case for Jetstream integration that you are referencing.
Can you share more about the use case where we need to use the persistent model of Kafka, Nats, and Fluvio at the same time?
Jetstream acks allows to ensure that fluvio got data successfully on NATS side. I didn't get, correct me if am wrong, but now NATS doesn't know, is all messages sent good or not.
Jetstream acks allows to ensure that fluvio got data successfully on NATS side. I didn't get, correct me if am wrong, but now NATS doesn't know, is all messages sent good or not.
I see what you are saying here it is for NATS to know if all the records made it to Fluvio. There may be an easier workaround for that. I will check on this and shape the work for when we focus on integrations.
Please feel free to contribute if you have the expertise.