Proposal: Streams DSL

[yzia2000]

Implement Streams DSL using Processor and storage API. Included abstractions:

1. KStream
2. KTable
3. KGroupedStream
4. KGroupedTable
5. WindowedJoins
6. PersistentJoins
7. Cogroups

This provides much needed abstractions stream processing.

If no effort has been made to tackle this and the proposal gets accepted, I would like to tackle this proposal.

[ubunatic]

Hey @yzia2000 we have nothing like that planned on our side. If you can come up with small PoC + examples to show how this is useful, we would be happy to review.

PS: your link leads to a blank page with an error Non-Image content-type returned

[yzia2000]

This is the link to the page: https://docs.confluent.io/platform/current/streams/developer-guide/dsl-api.html. It contains examples in java and provides a sql interface to process data with great guarantees.

[frairon]

Hi @yzia2000! Thanks for the proposal, the idea is great and in fact not that new. When we started writing goka we thought we will end up with a dsl too, but there really never was the need for it. Also the major reason is the lack of generics in go (which is about to change), that makes a dsl very cumbersome to use. If that changes, it's definitely worth another shot. So if you want to give it a try, you're very welcome to start something :)

[yzia2000]

After 1.18, maybe this can finally be discussed

[rupurt]

Generics are here :smile: Any updated thoughts on implementing these API's?