Multiple problems with the SQS implementation and documentation

[danielsitnik]

Hello, I just spent many hours trying to make the Amazon SQS integration work. I found some documentation and code issues that I'd like to share so we can make Mockintosh a better product.

1️⃣ Installing the "cloud" package

The documentation tells you to install the mockintosh[cloud] package to get the boto3 dependency. Copying/pasting the command shown on the documentation did not work on my mac, it only worked when I used quotes around the package name like this: pip3 install "mockintosh[cloud]".

Also, it is not clear if the package name is really mockintosh[cloud] or if it should be mockintosh[aws]. I tried both and none of them gave me an error.

2️⃣ Problem with the "markupsafe" dependency

After installing the cloud package like above, Mockintosh would no longer start and gave me this error:

ImportError: cannot import name 'soft_unicode' from 'markupsafe'

It could be a problem with one of the AWS dependencies or maybe Jinja2, there are issues like this one mentioning it. Anyway, I had to force a downgrade of MarkupSafe with pip3 install MarkupSafe==2.0.1 to make it work.

3️⃣ Using credentials (like secret_access_key) with special characters in them

My secret access key has a slash in it, just like this: gvmooa/xxxxxxxxx. This messes up the "address" field of the configuration file and some parts of the code that use urlparse will get wrong values, like this one.

I see two ways to solve this:

1. The way I solved it: just use the environment variables that are already mentioned in the documentation, and remove the credentials from the address field. However, the documentation should make it clear that this is the only way to work with credentials that have special characters like /, :, ?, @ or #.

2. Change the implementation (and make it clear in the documentation) so we can provide url encoded credentials in the address field. Eg: we would use %2F instead of / in the address and urllib.parse will work correctly. However, you will also need to change the implementation to unquote these values:

# import unquote: from urllib.parse import unquote
# in the _get_resource method:
aws_secret_access_key=environ.get('AWS_SECRET_ACCESS_KEY', '' if parsed.password is None else unquote(parsed.password)),
aws_access_key_id=environ.get('AWS_ACCESS_KEY_ID', '' if parsed.username is None else unquote(parsed.username)),

4️⃣ Queue must be FIFO and named with a ".fifo" suffix.

The last struggle was trying to understand why I was getting errors telling me that the queue was not found. Bear in mind that none of this is mentioned in the documentation. 🤦

I had to look at your source code (this line) to understand that you expect the queue name to have a .fifo suffix in AWS, but not in the configuration file. So if I have queue: myQueue in the configuration file, the actual name of the queue in AWS must be myQueue.fifo.

This also led me to understand that the SQS queues should be of the FIFO type, instead of the Standard one. This is also not mentioned anywhere in the documentation.