amazon-sqs-java-extended-client-lib icon indicating copy to clipboard operation
amazon-sqs-java-extended-client-lib copied to clipboard

Client adds a message attribute name to ReceiveMessageRequest for every call

Open dsw2127 opened this issue 7 years ago • 2 comments

AmazonSQSExtendedClient.receiveMessage(ReceiveMessageRequest) always appends SQSExtendedClientConstants.RESERVED_ATTRIBUTE_NAME to the list of message attribute names of the request passed in. Therefore if the request object is re-used for multiple invocations, this request can grow and grow. Eventually, if allowed to grow over time, SQS will deny the request with a 413 Request Entity Too Large. This seems to happen when the header gets to be somewhere around 1MB.

com.amazonaws.services.sqs.model.AmazonSQSException: null (Service: AmazonSQS; Status Code: 413; Error Code: 413 Request Entity Too Large; Request ID: null)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
        at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1792)
        at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1768)
        at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1374)
        at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1350)
        at com.amazon.sqs.javamessaging.AmazonSQSExtendedClientBase.receiveMessage(AmazonSQSExtendedClientBase.java:217)
        at com.amazon.sqs.javamessaging.AmazonSQSExtendedClient.receiveMessage(AmazonSQSExtendedClient.java:349)

Example of message attribute names over four invocations:

MessageAttributeNames: [SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize]

dsw2127 avatar Nov 27 '17 19:11 dsw2127

Workaround is to not re-use (i.e. create a new) ReceiveMessageRequest object for every invocation.

dsw2127 avatar Nov 27 '17 20:11 dsw2127

Hi there! Thanks for contributing this fix. We have pulled it into the latest release: https://github.com/awslabs/amazon-sqs-java-extended-client-lib/releases/tag/1.0.2

robin-aws avatar Sep 12 '18 21:09 robin-aws