inspectit-ocelot icon indicating copy to clipboard operation
inspectit-ocelot copied to clipboard

[Bug] - AgentCommandService throws "Read timed out" exception after agent commands have been disabled

Open heiko-holz opened this issue 2 years ago • 1 comments

Describe the bug When I enable the agent commands (and/or log-preloading) in the configuration server and then later disable them, I receive the following exception:

2022-11-15 11:40:51,139 ERROR 382527 --- [inspectIT] [pectit-thread-1] r.i.o.core.command.HttpCommandFetcher    : An error occurred while fetching an agent command.
java.net.SocketTimeoutException: Read timed out
	at java.base/java.net.SocketInputStream.socketRead0(Native Method)
	at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at rocks.inspectit.ocelot.core.command.HttpCommandFetcher.fetchCommand(HttpCommandFetcher.java:129)
	at rocks.inspectit.ocelot.core.command.CommandHandler.nextCommand(CommandHandler.java:72)
	at rocks.inspectit.ocelot.core.command.CommandHandler.nextCommand(CommandHandler.java:57)
	at rocks.inspectit.ocelot.core.command.AgentCommandService.run(AgentCommandService.java:93)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

The exceptions are also thrown for a while, even if I re-enable the agent commands.

To Reproduce Steps to reproduce the behavior:

  1. enable the agent commands
  2. wait until the agent commands are enabled (e.g., wait until you can get the logs from the Agent Status Table)
  3. disable the agent commands
  4. See error

Expected behavior I would not expect exceptions to be thrown if I disable agent commands

heiko-holz avatar Nov 15 '22 10:11 heiko-holz

While trying to find the cause of the problem, I figured out that the problem lays most likely at the "Show Logs" button. It is possible for me to switch the agent commands on/off without any error message. Then I noticed that the above error message always gets thrown approximately 30 seconds after pushing the "Show Logs" button.

danipaniii avatar Nov 15 '22 15:11 danipaniii