S3ToStepFunctions: add a SQS FIFO Queue to get exactly-once delivery

[joshwand]

In using this construct I found I was spending a lot more than expected because I was getting many duplicate events, and running my ML step function is quite expensive. I suggest adding a FIFO queue to get exactly-once delivery semantics (or change to an S3-SNS-SQS-Stepfunction workflow).

This is a :rocket: Feature Request

[biffgaut]

Thanks - we'll take a look.

Just to get some additional info in advance, if we were to add a FIFO capability to S3-SQS, then create an SQS-StepFunctions construct, would that work for your use case?

[joshwand]

Since I need something right away I'll be writing my own construct. While your suggestion does sound like it would work, IMO it would be better added to this construct, since to me the duplicate deliveries of S3 events is an undocumented and undesirable (and expensive!) behavior.

[biffgaut]

That's a good argument for it.