[💡 FEATURE REQUEST]: add support for multiple broker connections

[shieldz80]

Plugin

JOBS

I have an idea!

Hi,

Million thanks for the awesome project!

In my projects at work, I need to connect to different brokers in certain scenarios. For example I need to consume messages from one broker that is assigned to my team, and then during the processing of messages I need to push messages to another broker that is assigned to another team.

I was wondering if perhaps something like this would make sense (using amqp as an example):

amqp:
    # connection name
    brokerA:
        addr: amqp://guest:guest@brokerA:5672/
        tls:
            key: ""
            cert: ""
            root_ca: ""
            client_auth_type: no_client_certs
    # connection name
    brokerB:
        addr: amqp://guest:guest@brokerB:5672/
        tls:
            key: ""
            cert: ""
            root_ca: ""
            client_auth_type: no_client_certs
            
jobs:
    pipelines:
        pipeline-1:
            driver: amqp
            config:
                # this would be a new option
                connection: brokerA
        pipeline-2:
            driver: amqp
            config:
                # this would be a new option
                connection: brokerB