broadway_kafka icon indicating copy to clipboard operation
broadway_kafka copied to clipboard

Support for pipelines using Registry / via tuple

Open pbrooko opened this issue 3 years ago • 2 comments

Hello!

I'm busy playing with Broadway for a use-case with many pipeline's that are created & destroyed dynamically. The changes introduced in dashbitco/broadway#239 seem perfect for this by allowing pipeline naming using a via tuple / Registry.

The broadway_kafka producer still assumes pipeline's are named using an atom, and tries to use Module.concat/2 to build it's own child process names.

Is there a sane way to get the Kafka producer to also support naming its children using the pipeline's process_name/2 function?

pbrooko avatar Jun 30 '21 08:06 pbrooko

Yes, if we depend on master we should be able to rely on the process_name API for the same result. A PR would be welcome!

josevalim avatar Jun 30 '21 08:06 josevalim

Awesome, will see what I can put together..

pbrooko avatar Jun 30 '21 18:06 pbrooko