Connection issue with proxy in Java

[ghost]

Hello,

I have an issue when connecting to the elasticmq queue when on my work proxy. It seems to work fine if I disconnect from the internet. But otherwise I get the following.

Exception

[Fatal Error] loose.dtd:31:3: The declaration for the entity "HTML.Version" must end with '>'.
2016-08-23 12:44:10.282  INFO 14543 --- [pool-1-thread-1] c.a.http.DefaultErrorResponseHandler     : Unable to parse HTTP response content.

org.xml.sax.SAXParseException: The declaration for the entity "HTML.Version" must end with '>'.
    at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257) ~[na:1.8.0_73]
    at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339) ~[na:1.8.0_73]
    at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) ~[na:1.8.0_73]
    at com.amazonaws.util.XpathUtils.documentFrom(XpathUtils.java:133) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.util.XpathUtils.documentFrom(XpathUtils.java:140) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.DefaultErrorResponseHandler.parseContentAsXml(DefaultErrorResponseHandler.java:104) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.DefaultErrorResponseHandler.createAse(DefaultErrorResponseHandler.java:82) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.DefaultErrorResponseHandler.handle(DefaultErrorResponseHandler.java:69) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.DefaultErrorResponseHandler.handle(DefaultErrorResponseHandler.java:45) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1355) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:939) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:714) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:465) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:427) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:376) [aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1750) [aws-java-sdk-sqs-1.11.28.jar:na]
    at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1720) [aws-java-sdk-sqs-1.11.28.jar:na]
    at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1370) [aws-java-sdk-sqs-1.11.28.jar:na]
    at uk.co.bbc.seg_uploader.messaging.SQSSegNotificationQueue.retrieveSQSMessages(SQSSegNotificationQueue.java:59) [classes/:na]
    at uk.co.bbc.seg_uploader.messaging.SQSSegNotificationQueue.receiveNotificationsFromQueue(SQSSegNotificationQueue.java:48) [classes/:na]
    at uk.co.bbc.seg_uploader.messaging.SQSSegNotificationQueue.pollQueue(SQSSegNotificationQueue.java:35) [classes/:na]
    at uk.co.bbc.seg_uploader.schedule.SegUploaderScheduler.schedule(SegUploaderScheduler.java:50) [classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_73]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_73]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_73]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_73]
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) [spring-context-4.3.2.RELEASE.jar:4.3.2.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-4.3.2.RELEASE.jar:4.3.2.RELEASE]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_73]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_73]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_73]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_73]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_73]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_73]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73]

2016-08-23 12:44:10.298 ERROR 14543 --- [pool-1-thread-1] u.c.b.s.m.SQSSegNotificationQueue        : SQS Client failed to receive messages from Seg Uploader Queue

com.amazonaws.services.sqs.model.AmazonSQSException: null (Service: AmazonSQS; Status Code: 504; Error Code: 504 Gateway Time-out; Request ID: null)
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1386) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:939) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:714) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:465) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:427) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:376) ~[aws-java-sdk-core-1.11.28.jar:na]
    at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1750) ~[aws-java-sdk-sqs-1.11.28.jar:na]
    at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1720) ~[aws-java-sdk-sqs-1.11.28.jar:na]
    at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1370) ~[aws-java-sdk-sqs-1.11.28.jar:na]
    at uk.co.bbc.seg_uploader.messaging.SQSSegNotificationQueue.retrieveSQSMessages(SQSSegNotificationQueue.java:59) [classes/:na]
    at uk.co.bbc.seg_uploader.messaging.SQSSegNotificationQueue.receiveNotificationsFromQueue(SQSSegNotificationQueue.java:48) [classes/:na]
    at uk.co.bbc.seg_uploader.messaging.SQSSegNotificationQueue.pollQueue(SQSSegNotificationQueue.java:35) [classes/:na]
    at uk.co.bbc.seg_uploader.schedule.SegUploaderScheduler.schedule(SegUploaderScheduler.java:50) [classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_73]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_73]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_73]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_73]
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) [spring-context-4.3.2.RELEASE.jar:4.3.2.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-4.3.2.RELEASE.jar:4.3.2.RELEASE]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_73]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_73]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_73]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_73]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_73]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_73]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73]

Code

AWSCredentialsProviderChain credentialsProvider = new AWSCredentialsProviderChain(new DefaultAWSCredentialsProviderChain());
ClientConfiguration clientConfiguration = new ClientConfiguration();

AmazonSQSClient sqsClient = new AmazonSQSClient(credentialsProvider, clientConfiguration);
sqsClient.setEndpoint("http://localhost:9324/queue");
ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest("queue1").withMaxNumberOfMessages(10);

List<Message> sqsMessages;

ReceiveMessageResult receiveMessageResult = sqsClient.receiveMessage(receiveMessageRequest);
sqsMessages = receiveMessageResult.getMessages();

I had a similar issue with moto in Python and it was suggested to set env no_proxy to 169.254.169.254. https://github.com/spulec/moto/issues/605

os.environ['no_proxy'] = '169.254.169.254'

I tried this here to no avail. System.setProperty("no_proxy", "169.254.169.254");

Further debugging. Looks likes Apache is trying to route the request to: http://localhost.bbc.co.uk:9324/queue/queue1

Wondered if you had any ideas. Cheers

[ghost]

This is probably actually more of a AWS Java SQSClient isssue rather than elasticmq. May take up with AWS, unless you have any ideas

[adamw]

sorry, no idea :)

[ghost]

So it works if I add localhost to my proxy bypass in network settings. Just need to work out how to do in Java! Completely unrelated to elasticmq, sorry!

[mskonovalov]

The problem not in SQS client. I run elasticmq in docker and connect to it with 172.x.x.x:random_port. When I invoke sqsClient.createQueue(queueName) it returns me url starting from "http://localhost:9324" So it disregards Origin while generating URL

[adamw]

@mskonovalov There's a special setting for that: node_address (see the readme)

[mskonovalov]

@adamw and what to do if I have 2 clients and they should use 2 different domain names for the service?

[adamw]

well that wouldn't be supported right now ... but of course suggestions are welcome :)

[mskonovalov]

Sure, this problem should have common solution. I'd prefer to find time and maybe make a PR.

[adamw]

Would be awesome :)