URL in AWS output is not interpolated

[rtkkroland]

trafficstars

When trying to route messages based on metadata to different SQS endpoints, modifying the URL is the easiest, but it's the only field that's not interpolated :(

https://github.com/benthosdev/benthos/blame/34777e87934cba354219d803b32e26cfe565c930/internal/impl/aws/output_sqs.go#L39

This results in much sadness.

[krdenison]

Looking to do something like this in the url field where route is added to metadata by a previous processor and can be one of up to 20 values.

    output:
      aws_sqs:
        url: 'https://sqs.us-west-2.amazonaws.com/AWS_ACCOUNT/${! meta("route") }'
        max_in_flight: 64

[Jeffail]

Hey, this is technically possible as we can specify the URL per batch send request. However, one minor annoyance is that when you batch messages up (which is recommended for performance reasons) each individual message could yield a different URL, and so the logic and predictability of batch dispatching becomes much more complicated.

[rtkkroland]

Would having URL determined per message be worse/better performance than using the switch output?

[mihaitodor]

@rtkkroland: Not really. Just that the switch output requires that you to know the URLs when writing the config while interpolation allows you to extract the URL from each message at runtime. Benthos still has to extract some field from the message in each switch case to figure out which output to route the message to, so that will incur a similar performance penalty.

[rtkkroland]

Awesome, let's do URL interpolation then! :D

• Similar performance
• Run-time configurable