Fifo messages aren't deduped

[anderson-dan-w]

Anaconda-IPython sys.version[:5] == '2.7.6' boto3.__version__ == '1.5.17' moto.__version__ == (u'1.2.0',)

Reproduce

import boto3
from moto import mock_sqs

@mock_sqs
def dedupe_fifo():
    sqs = boto3.resource('sqs', region_name='us-west-2')
    queue = sqs.create_queue(
        QueueName='my-queue.fifo',
        Attributes={
            'FifoQueue': 'true',
        },
    )

    for _ in range(5):
        queue.send_message(
            MessageBody="test",
            MessageDeduplicationId="1",
            MessageGroupId="2",
        )
    assert queue.attributes['ApproximateNumberOfMessages'] == 1

Expected Behavior

For the assert to pass, as messages have been deduped. Note that my code sample should work whether Content-based deduplication is enabled or not, since the MessageBody is identical

Experienced Behavior

No de-duping occurs, so I get an AssertionError, assert 5 == 1, where 5 is length of my for-loop

[DHager]

Possibly related, when I retrieve messages from a mocked FIFO queue, the MessageGroupId is not present.

[spulec]

Thanks for opening. Marking this as a feature request.

[dstockstad]

Also encountered this so just wanted to +1

[rafcio19]

@bblommers let me take a look at this

[rafcio19]

nvm, looks like it's already been tackled https://github.com/getmoto/moto/blob/master/moto/sqs/models.py#L533

[bblommers]

@rafcio19 I'd have to verify this against AWS, but from my understanding of the docs and our implementation, two scenarios are not yet implemented in Moto:

• with ContentBasedDeduplication can be set to False (like in the example above) - we should still deduplicate. (The if-statement you linked is only active if ContentBasedDeduplication is set to True)
• the deduplication_id-attribute is set when ContentBasedDeduplication=True (https://github.com/getmoto/moto/pull/3538/files) - I assume that it only needs to be set if it isn't explicitly provided by the user

[rafcio19]

Gotcha, in that case let me take a look 👀