Working topic should have parameterizable names.

[NicholasJallan]

trafficstars

Is your feature request related to a problem? Please describe. The company that i'm working in has strict rules regarding topic names. Auto-creation of topics is not allowed, and names vary depending on the environment and other factors. tumbling_window functions use dedicated topics intentionally, but their names are not configurable.

Describe the solution you'd like Any function requiring a working topic should have a parameter to specify the topic name.

Describe alternatives you've considered Currently, there is no solution, which prevents the use of any function relying on a working topic—this represents a significant limitation.

[ghost]

Hey, thanks again for a detailed explanation.

To give a quick overview, Changelog topics are managed by the application.
For example, they always include the consumer group names and other parameters to ensure the consistency between local states and changelogs. Specifying full changelog topic names could result in incorrect use and data issues if the app uses the wrong topic for recovery.

Would it help with your problem if you could customize certain parts of the changelog topic names? Like changing the changelog__ prefix to match the conventions?

[NicholasJallan]

Hello Daniil,

Thanks for your answer. In the company where i'm working, the environnement is the prefix (dev-xxxx , sta-xxx, prod-xxx ) and as previously mentionned, all topics have to be created by an external routine/tool (auto-create function is not possible). A clean solution for me could also be to have a global config file on which quixstreams would rely, explaining how the working topic name should be computed, especially with pamaterizable prefix and suffix, and maybe global length. This file could be helpful to possibly address further usecases later on not related to topic naming.

So, to reply your question : yes it would probably help if the names where customizable so we can match the conventions.

[francoishornoy]

+1 for being able to customize the changelog topic names.

In our case, after using sdf.concat(other_sdf) to concatenate several topics together, the resulting changelog topic name exceeds the 255 char length limit.