SEVERE: Inference failed on 3th retry

[yangkl96]

I am trying to run the example at https://github.com/triton-inference-server/client/blob/main/src/java/src/main/java/triton/client/examples/SimpleInferPerf.java, but I keep getting the following error:

Sep 20, 2023 3:26:44 PM triton.client.InferenceServerClient infer
SEVERE: Inference failed on 3th retry, url: http://0.0.0.0:8000/v2/models/simple/infer
java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: no further information
	at org.apache.http.concurrent.BasicFuture.getResult(BasicFuture.java:71)
	at org.apache.http.concurrent.BasicFuture.get(BasicFuture.java:102)
	at org.apache.http.impl.nio.client.FutureWrapper.get(FutureWrapper.java:75)
	at triton.client.InferenceServerClient.infer(InferenceServerClient.java:297)
	at Triton.lambda$main$2(Triton.java:86)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.ConnectException: Connection refused: no further information
	at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	... 1 more

[the-david-oy]

CC: @krishung5

[krishung5]

@yangkl96 Sorry for the late reply. Could you provide the commands/minimum steps to reproduce the error? There seems to be some connection issue, I was wondering if you are running the Java client within the same container, or a separate container.

[yangkl96]

No worries. I am running this code within IntelliJ IDEA triton_error.txt

I have a pom.xml from here: https://github.com/triton-inference-server/client/blob/main/src/java/pom.xml

[the-david-oy]

It's going to be hard for us to reproduce this with just the code. Are you otherwise able to ping localhost:8000 with the code or outside of it? Is that port accessible to the Java program?

That would be a good sanity test that the server is stood up with the right port exposed, and that the port is accessible to the client code.