scouter icon indicating copy to clipboard operation
scouter copied to clipboard

Scouter Metrices in Client are showing disconnected in xlogs

Open sdevkishanjoshi opened this issue 2 years ago • 0 comments

Hello, We have deployed scouter in EKS Version 1.25.

We are getting disconnection in Scouter Xlogs Metrices when trying to view from Scouter Client. Upon checking We are getting some exceptions, below are some logs of the same.

java.net.SocketException: Operation timed out (Write failed)
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
        at java.io.DataOutputStream.write(DataOutputStream.java:107)
        at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
        at scouter.io.DataOutputX.write(DataOutputX.java:408)
        at scouter.server.netio.service.handle.XLogService$$anonfun$getRealtimePerfGroup$2.apply(XLogService.scala:203)
        at scouter.server.netio.service.handle.XLogService$$anonfun$getRealtimePerfGroup$2.apply(XLogService.scala:201)
        at scouter.server.util.EnumerScala$.forward(EnumerScala.scala:48)
        at scouter.server.netio.service.handle.XLogService.getRealtimePerfGroup(XLogService.scala:201)
        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at scouter.server.netio.service.ServiceHandlingProxy$Invocation.exec(ServiceHandlingProxy.java:42)
        at scouter.server.netio.service.ServiceHandlingProxy.process(ServiceHandlingProxy.java:100)
        at scouter.server.netio.service.net.ServiceWorker.run(TcpServiceWorker.scala:126)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
java.net.SocketException: Broken pipe (Write failed)
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
        at java.io.DataOutputStream.write(DataOutputStream.java:107)
        at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
        at scouter.io.DataOutputX.write(DataOutputX.java:408)
        at scouter.server.netio.service.handle.XLogService$$anonfun$getRealtimePerfGroup$2.apply(XLogService.scala:203)
        at scouter.server.netio.service.handle.XLogService$$anonfun$getRealtimePerfGroup$2.apply(XLogService.scala:201)
        at scouter.server.util.EnumerScala$.forward(EnumerScala.scala:48)
        at scouter.server.netio.service.handle.XLogService.getRealtimePerfGroup(XLogService.scala:201)
        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at scouter.server.netio.service.ServiceHandlingProxy$Invocation.exec(ServiceHandlingProxy.java:42)
        at scouter.server.netio.service.ServiceHandlingProxy.process(ServiceHandlingProxy.java:100)
        at scouter.server.netio.service.net.ServiceWorker.run(TcpServiceWorker.scala:126)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
java.net.SocketException: Connection reset
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
        at java.io.DataOutputStream.write(DataOutputStream.java:107)
        at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
        at scouter.io.DataOutputX.write(DataOutputX.java:408)
        at scouter.server.netio.service.handle.XLogService$$anonfun$getRealtimePerfGroup$2.apply(XLogService.scala:203)
        at scouter.server.netio.service.handle.XLogService$$anonfun$getRealtimePerfGroup$2.apply(XLogService.scala:201)
        at scouter.server.util.EnumerScala$.forward(EnumerScala.scala:48)
        at scouter.server.netio.service.handle.XLogService.getRealtimePerfGroup(XLogService.scala:201)
        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at scouter.server.netio.service.ServiceHandlingProxy$Invocation.exec(ServiceHandlingProxy.java:42)
        at scouter.server.netio.service.ServiceHandlingProxy.process(ServiceHandlingProxy.java:100)
        at scouter.server.netio.service.net.ServiceWorker.run(TcpServiceWorker.scala:126)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Scouter Client Version: 2.17.1

From infra prospective things look fine it has enough RAM & CPU to consume. Let us know if we are missing some configuration.

image

sdevkishanjoshi avatar Oct 03 '23 07:10 sdevkishanjoshi