Fix max throughput issue

[tavakara]

This pull request fixes the issue report in https://github.com/qubole/s3-sqs-connector/issues/9.

[crojasqnextroll]

Hi @tavakara this PR makes sense to me, were you able to get a better throughput rate with this change?

[abhishekd0907]

@tavakara SQSClient doesn't fetch one message per request but a list of messages. However, due to the current code, there is a minimum delay of one second between two requests. I am happy to accept a change that will make this delay configurable in milliseconds as compared to seconds.

However, I would like to point out that the advisable way of polling SQS is to minimize the frequency of polling and fetch the maximum possible messages per request in order to save costs. Also, keep in mind that polling SQS too frequently by your client may lead to throttling and give 429 errors as discussed here Stackoverflow

[sujeet-srivastava]

I changed the sqsFetchInterval to accept milliseconds: see here - https://github.com/sujeet-srivastava/s3-sqs-connector

Tested as below:

• Added 4890 (total size =10Gb) files to a bucket linked up to a Queue
• Created a streaming sql query from this Queue to dump the content into another location after minor processing
• I could achieve a message receive rate at 3.6k messages/sec in SQS with fetchInterval set to 10 millisec
• did not see throttling
• no data loss.
• The limit to seconds seems unnecessary

This fix is needed