pinpoint
pinpoint copied to clipboard
Issue with Concurrent Execution of Multiple APMs Due to Module Permissions(타 APM 동시 실행 시 Module 권한 관련 이슈.)
Prerequisites
Please check the FAQ, and search existing issues for similar questions before creating a new issue.YOU MAY DELETE THIS PREREQUISITES SECTION.
What version of pinpoint are you using?
master/v1.7.3/v1.7.2 etc.
Pinpoint Collector, Agent, Web Version: 2.3.3 and compatible versions Instrumentation Server: Java 9 and above (specifically tested on 11, 17, and 21, with logs from version 21)
핀포인트 Collector, Agent, Web 버전 : 2.3.3 및 상응 호환 버전
계측 서버 : Java 9 이상 (11, 17, 21), 로그는 21버전 실행 시 발생
Describe your problem**
Dear Support Team,
I am writing to inquire about an issue we are encountering with the concurrent use of multiple APM tools. Currently, we are using Pinpoint alongside another asm, bytecode manipulation monitoring agent, Opentelemetry, which is based on ByteBuddy.
Issue Summary:
Java 8:
When building with the same -javaagent options, there are no issues. Both agents run successfully and perform their respective instrumentation tasks without any conflicts.
Java 9 and above (Java 11, 17, 21):
Concurrent builds fail for Pinpoint, although Opentelemetry continues to run successfully. From the logs, it appears that the failure is due to the inability to obtain necessary module permissions (unnamed module). .. particularly seems to be an issue related to Netty and Grpc.
We suspect this issue arises from module permissions required by Pinpoint in Java versions 9 and above. We would like to know if there are any configuration settings or build options within Pinpoint that can resolve this issue.
Attached are the build scripts and error logs for your reference. Thank you for your assistance.
안녕하세요. APM 사용 이슈 관련 문의드립니다.
현재 Pinpoint와 유사한 ASM, ByteBuddy 기반의 바이트코드 조작 모니터링 Agent를 병행하여 사용중입니다. (Opentelemetry)
Java 8의 경우.
빌드 시 동일한 -javaagent 옵션을 통해 빌드해도 문제가 생기지 않습니다.
정상적으로 잘 실행되며 각각 계측이 가능합니다.
Java 9 이상 버전의 경우 (Java 11, 17, 21)
동시 빌드의 경우 Pinpoint의 빌드가 실패합니다. (타 Apm- opentelemetry의 경우는 실행됩니다.)
로그를 확인해 본 결과, Module 권한을 얻지 못해 발생한 일로 추측됩니다. (unnamed module) .. 특히 Netty와 Grpc 관련 이슈로 보입니다.
핀포인트에서 Configue 설정이나 빌드 시 설정을 통해 해당 이슈를 해결할 수 있는지 문의드립니다.
빌드 스크립트와 에러 로그를 첨부합니다. 읽어 주셔서 감사합니다.
Logs
에러 로그(Error log)
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[otel.javaagent 2024-06-20 11:23:15:294 +0900] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 2.4.0
06-20 11:23:17.749 INFO PinpointBootStrap : pinpoint agentArgs:null
06-20 11:23:17.750 INFO PinpointBootStrap : PinpointBootStrap.ClassLoader:null
06-20 11:23:17.750 INFO PinpointBootStrap : ContextClassLoader:jdk.internal.loader.ClassLoaders$AppClassLoader@4e0e2f2a
06-20 11:23:17.755 INFO ClassAgentPathFinder : agentPath:/opt/agent.pinpoint/pinpoint-bootstrap-2.3.3.jar
06-20 11:23:17.755 INFO PinpointBootStrap : JavaAgentPath:/opt/agent.pinpoint/pinpoint-bootstrap-2.3.3.jar
06-20 11:23:17.756 INFO AgentDirBaseClassPathResolver : Agent original-path:/opt/agent.pinpoint
06-20 11:23:17.757 INFO AgentDirBaseClassPathResolver : Agent canonical-path:/opt/agent.pinpoint
06-20 11:23:17.758 INFO BootDir : found pinpoint-commons.jar path:/opt/agent.pinpoint/boot/pinpoint-commons-2.3.3.jar
06-20 11:23:17.758 INFO BootDir : found pinpoint-bootstrap-core.jar path:/opt/agent.pinpoint/boot/pinpoint-bootstrap-core-2.3.3.jar
06-20 11:23:17.770 INFO BootDir : found pinpoint-annotations.jar path:/opt/agent.pinpoint/boot/pinpoint-annotations-2.3.3.jar
06-20 11:23:17.770 INFO BootDir : found pinpoint-bootstrap-java8.jar path:/opt/agent.pinpoint/boot/pinpoint-bootstrap-java8-2.3.3.jar
06-20 11:23:17.771 INFO BootDir : found pinpoint-bootstrap-java9.jar path:/opt/agent.pinpoint/boot/pinpoint-bootstrap-java9-2.3.3.jar
06-20 11:23:17.775 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-agentsdk-async-plugin-2.3.3.jar
06-20 11:23:17.775 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-common-servlet-2.3.3.jar
06-20 11:23:17.775 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-httpclient3-plugin-2.3.3.jar
06-20 11:23:17.775 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-httpclient4-plugin-2.3.3.jar
06-20 11:23:17.775 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-ning-asynchttpclient-plugin-2.3.3.jar
06-20 11:23:17.776 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-resttemplate-plugin-2.3.3.jar
06-20 11:23:17.776 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-netty-plugin-2.3.3.jar
06-20 11:23:17.776 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-gson-plugin-2.3.3.jar
06-20 11:23:17.776 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-cubrid-jdbc-driver-plugin-2.3.3.jar
06-20 11:23:17.776 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-informix-jdbc-driver-plugin-2.3.3.jar
06-20 11:23:17.777 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-mysql-jdbc-driver-plugin-2.3.3.jar
06-20 11:23:17.777 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-mariadb-jdbc-driver-plugin-2.3.3.jar
06-20 11:23:17.777 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-mssql-jdbc-driver-plugin-2.3.3.jar
06-20 11:23:17.777 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-postgresql-jdbc-driver-plugin-2.3.3.jar
06-20 11:23:17.777 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-jtds-plugin-2.3.3.jar
06-20 11:23:17.778 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-oracle-jdbc-driver-plugin-2.3.3.jar
06-20 11:23:17.778 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-commons-dbcp-plugin-2.3.3.jar
06-20 11:23:17.778 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-commons-dbcp2-plugin-2.3.3.jar
06-20 11:23:17.778 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-hikaricp-plugin-2.3.3.jar
06-20 11:23:17.778 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-cassandra-driver-plugin-2.3.3.jar
06-20 11:23:17.779 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-jdk-http-plugin-2.3.3.jar
06-20 11:23:17.779 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-redis-plugin-2.3.3.jar
06-20 11:23:17.779 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-redis-lettuce-plugin-2.3.3.jar
06-20 11:23:17.779 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-redis-redisson-plugin-2.3.3.jar
06-20 11:23:17.779 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-tomcat-plugin-2.3.3.jar
06-20 11:23:17.780 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-json-lib-plugin-2.3.3.jar
06-20 11:23:17.780 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-jackson-plugin-2.3.3.jar
06-20 11:23:17.780 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-thrift-plugin-2.3.3.jar
06-20 11:23:17.780 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-user-plugin-2.3.3.jar
06-20 11:23:17.780 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-arcus-plugin-2.3.3.jar
06-20 11:23:17.789 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-google-httpclient-plugin-2.3.3.jar
06-20 11:23:17.789 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-grpc-plugin-2.3.3.jar
06-20 11:23:17.790 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-jetty-plugin-2.3.3.jar
06-20 11:23:17.790 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-websphere-plugin-2.3.3.jar
06-20 11:23:17.790 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-spring-plugin-2.3.3.jar
06-20 11:23:17.790 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-spring-boot-plugin-2.3.3.jar
06-20 11:23:17.790 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-spring-webflux-plugin-2.3.3.jar
06-20 11:23:17.791 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-ibatis-plugin-2.3.3.jar
06-20 11:23:17.791 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-mybatis-plugin-2.3.3.jar
06-20 11:23:17.791 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-okhttp-plugin-2.3.3.jar
06-20 11:23:17.791 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-log4j-plugin-2.3.3.jar
06-20 11:23:17.791 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-logback-plugin-2.3.3.jar
06-20 11:23:17.792 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-log4j2-plugin-2.3.3.jar
06-20 11:23:17.792 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-dubbo-plugin-2.3.3.jar
06-20 11:23:17.792 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-activemq-client-plugin-2.3.3.jar
06-20 11:23:17.792 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-cxf-plugin-2.3.3.jar
06-20 11:23:17.792 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-hystrix-plugin-2.3.3.jar
06-20 11:23:17.793 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-jboss-plugin-2.3.3.jar
06-20 11:23:17.793 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-vertx-plugin-2.3.3.jar
06-20 11:23:17.793 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-resin-plugin-2.3.3.jar
06-20 11:23:17.793 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-jsp-plugin-2.3.3.jar
06-20 11:23:17.793 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-rxjava-plugin-2.3.3.jar
06-20 11:23:17.794 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-rabbitmq-plugin-2.3.3.jar
06-20 11:23:17.794 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-weblogic-plugin-2.3.3.jar
06-20 11:23:17.795 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-akka-http-plugin-2.3.3.jar
06-20 11:23:17.797 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-undertow-plugin-2.3.3.jar
06-20 11:23:17.798 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-undertow-servlet-plugin-2.3.3.jar
06-20 11:23:17.798 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-kafka-plugin-2.3.3.jar
06-20 11:23:17.798 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-mongodb-driver-plugin-2.3.3.jar
06-20 11:23:17.798 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-fastjson-plugin-2.3.3.jar
06-20 11:23:17.798 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-druid-plugin-2.3.3.jar
06-20 11:23:17.798 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-hbase-plugin-2.3.3.jar
06-20 11:23:17.799 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-openwhisk-plugin-2.3.3.jar
06-20 11:23:17.799 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-elasticsearch-bboss-plugin-2.3.3.jar
06-20 11:23:17.799 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-apache-dubbo-plugin-2.3.3.jar
06-20 11:23:17.799 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-elasticsearch-plugin-2.3.3.jar
06-20 11:23:17.799 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-reactor-plugin-2.3.3.jar
06-20 11:23:17.800 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-reactor-netty-plugin-2.3.3.jar
06-20 11:23:17.800 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-jdk-completable-future-plugin-2.3.3.jar
06-20 11:23:17.800 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-thread-plugin-2.3.3.jar
06-20 11:23:17.800 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-process-plugin-2.3.3.jar
06-20 11:23:17.800 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-paho-mqtt-plugin-2.3.3.jar
06-20 11:23:17.800 INFO AgentDirBaseClassPathResolver : Found plugins:/opt/agent.pinpoint/plugin/pinpoint-rocketmq-plugin-2.3.3.jar
06-20 11:23:17.802 INFO PinpointBootStrap : appendToBootstrapClassLoader:/opt/agent.pinpoint/boot/pinpoint-commons-2.3.3.jar
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
06-20 11:23:17.803 INFO PinpointBootStrap : appendToBootstrapClassLoader:/opt/agent.pinpoint/boot/pinpoint-bootstrap-core-2.3.3.jar
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
06-20 11:23:17.803 INFO PinpointBootStrap : appendToBootstrapClassLoader:/opt/agent.pinpoint/boot/pinpoint-annotations-2.3.3.jar
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
06-20 11:23:17.803 INFO PinpointBootStrap : appendToBootstrapClassLoader:/opt/agent.pinpoint/boot/pinpoint-bootstrap-java8-2.3.3.jar
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
06-20 11:23:17.804 INFO PinpointBootStrap : appendToBootstrapClassLoader:/opt/agent.pinpoint/boot/pinpoint-bootstrap-java9-2.3.3.jar
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
06-20 11:23:17.804 INFO PinpointBootStrap : parentClassLoader:BootStrapClassLoader:null
06-20 11:23:17.805 INFO PinpointBootStrap : java9 module detected
06-20 11:23:17.805 INFO PinpointBootStrap : ModuleBootLoader start
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] pinpoint Module id:unnamed module @45f421c
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] pinpoint Module.isNamed:false
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] pinpoint Module.name:null
06-20 11:23:17.835 INFO IdValidator : check SystemProperties(-D) agentId:pinpoint_agent_consumer_1
06-20 11:23:17.837 INFO IdValidator : invalid Id. SystemProperties(-D) agentId can only contain [a-zA-Z0-9], '.', '-', '_'. maxLength:24 value:pinpoint_agent_consumer_1
06-20 11:23:17.838 INFO AgentIdResolver : Failed to resolve AgentId(-Dpinpoint.agentId)
06-20 11:23:17.839 INFO AgentIdResolver : Auto generate AgentId='hmly2lA2TxO9PLGqdrJmlQ'
06-20 11:23:17.839 INFO IdValidator : check SystemProperties(-D) applicationName:otel_consumer_java_1
06-20 11:23:17.839 INFO AgentIdResolver : SystemProperties(-D) pinpoint.applicationName=otel_consumer_java_1
06-20 11:23:17.839 INFO AgentIdResolver : No AgentName(-Dpinpoint.agentName) provided, it's optional!
06-20 11:23:17.849 INFO ProfilePropertyLoader : load default config:/opt/agent.pinpoint/pinpoint-root.config
06-20 11:23:17.858 INFO ProfilePropertyLoader : active profile:release
06-20 11:23:17.858 INFO ProfilePropertyLoader : load profile:/opt/agent.pinpoint/profiles/release/pinpoint.config
06-20 11:23:17.861 INFO ProfilePropertyLoader : load external config:/opt/agent.pinpoint/pinpoint-root.config
06-20 11:23:17.868 INFO ProfilePropertyLoader : agent root path:/opt/agent.pinpoint
06-20 11:23:17.869 INFO ProfilePropertyLoader : logConfig path:/opt/agent.pinpoint/profiles/release/
06-20 11:23:17.899 INFO PinpointStarter : logPath:/opt/agent.pinpoint/logs
06-20 11:23:17.899 INFO PinpointStarter : pinpoint version:2.3.3
06-20 11:23:17.899 INFO PinpointStarter : Log directory maxbackupsize=5
06-20 11:23:17.901 INFO PinpointStarter : agent JarPath:/opt/agent.pinpoint/pinpoint-bootstrap-2.3.3.jar
06-20 11:23:17.901 INFO PinpointStarter : agent LibDir:/opt/agent.pinpoint/lib
06-20 11:23:17.901 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-profiler-2.3.3.jar
06-20 11:23:17.901 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-thrift-2.3.3.jar
06-20 11:23:17.901 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-rpc-2.3.3.jar
06-20 11:23:17.902 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-plugins-loader-2.3.3.jar
06-20 11:23:17.902 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-profiler-logging-2.3.3.jar
06-20 11:23:17.902 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-profiler-optional-jdk7-2.3.3.jar
06-20 11:23:17.902 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-profiler-optional-jdk8-2.3.3.jar
06-20 11:23:17.902 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-profiler-optional-jdk9-2.3.3.jar
06-20 11:23:17.902 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-grpc-2.3.3.jar
06-20 11:23:17.902 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-agent-proxy-common-2.3.3.jar
06-20 11:23:17.903 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-agent-proxy-apache-plugin-2.3.3.jar
06-20 11:23:17.903 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-agent-proxy-app-plugin-2.3.3.jar
06-20 11:23:17.903 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-agent-proxy-nginx-plugin-2.3.3.jar
06-20 11:23:17.903 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-agent-proxy-user-plugin-2.3.3.jar
06-20 11:23:17.903 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-commons-buffer-2.3.3.jar
06-20 11:23:17.903 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/pinpoint-commons-profiler-2.3.3.jar
06-20 11:23:17.903 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/log4j2.component.properties
06-20 11:23:17.909 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-3.10.6.Final.jar
06-20 11:23:17.909 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/snakeyaml-1.27.jar
06-20 11:23:17.909 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/guava-30.1-android.jar
06-20 11:23:17.909 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/failureaccess-1.0.1.jar
06-20 11:23:17.909 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/jsr305-3.0.2.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/checker-compat-qual-2.5.5.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/error_prone_annotations-2.3.4.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/j2objc-annotations-1.3.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/guice-4.2.2.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/javax.inject-1.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/aopalliance-1.0.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/asm-9.1.jar
06-20 11:23:17.910 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/asm-commons-9.1.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/asm-util-9.1.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/asm-tree-9.1.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/asm-analysis-9.1.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/libthrift-0.12.0.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/grpc-core-1.36.2.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/grpc-api-1.36.2.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/grpc-context-1.36.2.jar
06-20 11:23:17.911 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/gson-2.8.6.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/annotations-4.1.1.4.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/animal-sniffer-annotations-1.19.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/perfmark-api-0.23.0.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/grpc-netty-1.36.2.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-handler-4.1.58.Final.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-common-4.1.58.Final.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-resolver-4.1.58.Final.jar
06-20 11:23:17.912 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-buffer-4.1.58.Final.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-transport-4.1.58.Final.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-codec-4.1.58.Final.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-transport-native-epoll-4.1.58.Final-linux-x86_64.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-transport-native-unix-common-4.1.58.Final.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-codec-http2-4.1.58.Final.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-codec-http-4.1.58.Final.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/grpc-stub-1.36.2.jar
06-20 11:23:17.913 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/grpc-protobuf-1.36.2.jar
06-20 11:23:17.914 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/protobuf-java-3.12.0.jar
06-20 11:23:17.914 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/proto-google-common-protos-2.0.1.jar
06-20 11:23:17.914 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/grpc-protobuf-lite-1.36.2.jar
06-20 11:23:17.914 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/netty-tcnative-boringssl-static-2.0.36.Final.jar
06-20 11:23:17.914 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/slf4j-api-1.7.30.jar
06-20 11:23:17.914 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/log4j-core-2.12.3.jar
06-20 11:23:17.914 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/log4j-api-2.12.3.jar
06-20 11:23:17.915 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/log4j-slf4j-impl-2.12.3.jar
06-20 11:23:17.915 INFO PinpointStarter : agent Lib:file:/opt/agent.pinpoint/lib/
06-20 11:23:17.915 INFO PinpointStarter : agent config:/opt/agent.pinpoint/pinpoint-root.config
06-20 11:23:17.957 INFO PinpointStarter : defineAgentModule
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] bootstrap unnamedModule:unnamed module @45f421c
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] platform unnamedModule:unnamed module @58af5076
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] system unnamedModule:unnamed module @1816e24a
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] defineModule classLoader: Java9ClassLoader{name=pinpoint.agent} com.navercorp.pinpoint.bootstrap.java9.classloader.Java9ClassLoader@7fd987ef
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] defineModule classLoader-unnamedModule: unnamed module @49469ffa
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] packages:[io.netty.handler.codec.rtsp, com.navercorp.pinpoint.profiler.context.compress, com.google.protobuf, com.navercorp.pinpoint.profiler.monitor.collector.activethread, com.navercorp.pinpoint.profiler.monitor.metric.cpu.ibm, org.apache.logging.log4j.core.config.plugins.validation.validators, io.grpc.protobuf.lite, io.netty.handler.codec.serialization, org.jboss.netty.buffer, io.netty.channel.socket.nio, com.navercorp.pinpoint.profiler.instrument.classreading, com.google.inject.internal.cglib.core, com.google.inject, com.navercorp.pinpoint.profiler.monitor.collector.cpu, io.netty.handler.codec.protobuf, com.navercorp.pinpoint.profiler.monitor.metric, com.navercorp.pinpoint.profiler.plugin.filter, org.jboss.netty.channel.socket.oio, io.netty.handler.flush, com.navercorp.pinpoint.profiler.context.provider.stat.cpu, org.yaml.snakeyaml.emitter, org.jboss.netty.channel.group, io.netty.handler.ssl, com.navercorp.pinpoint.profiler.monitor.metric.buffer, com.navercorp.pinpoint.thrift.dto, com.navercorp.pinpoint.profiler.instrument, org.apache.logging.log4j.core.layout, org.jboss.netty.handler.timeout, org.apache.logging.log4j.core.message, com.google.gson.internal, com.navercorp.pinpoint.profiler.shutdown, io.netty.util.internal, org.jboss.netty.handler.execution, org.apache.logging.log4j, org.jboss.netty.handler.ipfilter, org.apache.logging.log4j.core.appender.rolling.action, org.slf4j.event, org.apache.logging.log4j.core.tools.picocli, com.navercorp.pinpoint.profiler.context.provider.stat.filedescriptor, org.jboss.netty.util.internal.jzlib, org.apache.logging.log4j.core.appender.rewrite, org.jboss.netty.channel.socket.nio, com.google.common.base, io.netty.channel.internal, io.netty.handler.codec.spdy, org.jboss.netty.handler.codec.spdy, com.navercorp.pinpoint.profiler.jdbc, com.google.common.cache, com.google.errorprone.annotations, com.google.inject.internal.cglib.core.internal, com.navercorp.pinpoint.profiler.monitor.metric.cpu.oracle, org.jboss.netty.handler.codec.compression, org.jboss.netty.util, com.google.longrunning, org.apache.logging.log4j.core.appender.mom.kafka, com.navercorp.pinpoint.profiler.context.storage, org.apache.logging.log4j.core.config.yaml, com.google.rpc, org.slf4j, com.navercorp.pinpoint.profiler.monitor.metric.activethread, org.apache.logging.log4j.core.selector, com.navercorp.pinpoint.profiler.monitor.metric.uri, com.navercorp.pinpoint.grpc.client.interceptor, com.navercorp.pinpoint.profiler, com.navercorp.pinpoint.io.request, org.apache.logging.log4j.core.config.builder.impl, com.navercorp.pinpoint.profiler.sampler, org.apache.logging.slf4j, org.jboss.netty.handler.ssl, com.navercorp.pinpoint.profiler.plugin.config, io.grpc.internal, com.google.cloud.audit, org.apache.logging.log4j.core.config.plugins.validation.constraints, org.apache.logging.log4j.core.osgi, org.objectweb.asm, com.navercorp.pinpoint.rpc.server, com.google.common.math, io.netty.handler.codec.http, com.navercorp.pinpoint.profiler.context.annotation, com.navercorp.pinpoint.thrift.io, com.navercorp.pinpoint.thrift.dto.command, io.netty.handler.traffic, org.apache.logging.log4j.core.appender.rolling, io.grpc.stub.annotations, com.navercorp.pinpoint.rpc.codec, io.netty.handler.codec.http.cookie, com.google.rpc.context, com.google.api, com.google.common.base.internal, org.apache.thrift.async, io.netty.channel, org.slf4j.helpers, com.navercorp.pinpoint.profiler.context.provider.metric, com.navercorp.pinpoint.profiler.monitor.metric.filedescriptor.ibm, com.navercorp.pinpoint.grpc.client.config, org.apache.logging.log4j.core.config.plugins.convert, org.apache.logging.log4j.core.config.composite, com.navercorp.pinpoint.profiler.monitor.collector.deadlock, com.google.common.primitives, com.navercorp.pinpoint.profiler.plugin, org.codehaus.mojo.animal_sniffer, org.jboss.netty.channel.socket.http, com.google.type, com.navercorp.pinpoint.thrift.util, com.google.inject.name, com.navercorp.pinpoint.profiler.context.provider.thrift, com.navercorp.pinpoint.profiler.context.provider.sampler, com.navercorp.pinpoint.profiler.context.active, com.google.inject.matcher, org.apache.logging.log4j.core.appender, com.google.inject.internal, com.navercorp.pinpoint.profiler.instrument.classloading, com.navercorp.pinpoint.agent.plugin.proxy.apache, com.google.common.html, org.apache.logging.log4j.core.tools, org.apache.logging.log4j.core.net, org.apache.logging.log4j.core.appender.mom, com.navercorp.pinpoint.profiler.util, org.jboss.netty.handler.ssl.util, org.yaml.snakeyaml.composer, io.netty.handler.codec, com.google.gson.stream, org.jboss.netty.channel.socket, io.netty.handler.stream, io.netty.handler.codec.http.multipart, io.netty.channel.socket, org.apache.logging.log4j.core.config.status, com.navercorp.pinpoint.profiler.context.provider.stat.totalthread, com.google.common.escape, com.navercorp.pinpoint.common.buffer, com.google.thirdparty.publicsuffix, javax.annotation.meta, com.navercorp.pinpoint.profiler.context.module.config, com.navercorp.pinpoint.profiler.interceptor.scope, com.navercorp.pinpoint.rpc.common, com.navercorp.pinpoint.profiler.instrument.lambda, org.apache.thrift.scheme, org.yaml.snakeyaml.error, org.jboss.netty.container.osgi, com.navercorp.pinpoint.profiler.context.id, com.google.gson.annotations, com.navercorp.pinpoint.profiler.context.config, com.navercorp.pinpoint.rpc.packet.stream, org.yaml.snakeyaml, com.google.inject.multibindings, org.objectweb.asm.tree.analysis, io.netty.util.concurrent, io.netty.handler.codec.json, com.navercorp.pinpoint.profiler.monitor.metric.deadlock, com.navercorp.pinpoint.agent.plugin.proxy.user, com.navercorp.pinpoint.profiler.monitor.collector.loadedclass, com.navercorp.pinpoint.profiler.context.recorder, org.objectweb.asm.signature, org.apache.logging.log4j.core.util, org.apache.logging.log4j.core.script, com.navercorp.pinpoint.grpc.trace, com.navercorp.pinpoint.common.profiler.concurrent.jsr166, com.navercorp.pinpoint.rpc.server.handler, org.yaml.snakeyaml.tokens, com.navercorp.pinpoint.profiler.monitor.collector.jvmgc, com.navercorp.pinpoint.profiler.context.javamodule, com.navercorp.pinpoint.rpc, com.google.inject.internal.util, org.aopalliance.aop, io.netty.channel.unix, io.grpc, com.google.inject.util, com.navercorp.pinpoint.profiler.monitor.metric.filedescriptor.oracle, com.google.gson, com.navercorp.pinpoint.profiler.context.grpc, io.netty.handler.codec.http.websocketx.extensions.compression, org.yaml.snakeyaml.introspector, com.google.common.util.concurrent, org.jboss.netty.handler.codec.serialization, com.navercorp.pinpoint.rpc.control, org.jboss.netty.handler.codec.replay, com.google.common.eventbus, com.google.gson.internal.bind, com.navercorp.pinpoint.rpc.util, com.navercorp.pinpoint.profiler.context.provider.stat.jvmgc, com.google.common.collect, io.netty.buffer, io.netty.channel.embedded, org.apache.logging.log4j.core.async, com.navercorp.pinpoint.profiler.context.provider.stat.buffer, com.navercorp.pinpoint.profiler.metadata, android.annotation, com.navercorp.pinpoint.grpc.logging, com.navercorp.pinpoint.profiler.context, io.netty.handler.ipfilter, org.apache.logging.log4j.core.appender.routing, io.netty.handler.codec.string, io.netty.buffer.search, javax.inject, com.navercorp.pinpoint.rpc.cluster, com.navercorp.pinpoint.grpc, org.jboss.netty.channel, com.navercorp.pinpoint.profiler.context.module, com.navercorp.pinpoint.io, com.google.common.net, com.navercorp.pinpoint.profiler.monitor.metric.gc, com.navercorp.pinpoint.loader.plugins, io.netty.channel.socket.oio, com.google.common.util.concurrent.internal, com.navercorp.pinpoint.grpc.util, com.navercorp.pinpoint.io.util, com.navercorp.pinpoint.profiler.context.provider.stat.loadedclass, com.google.gson.internal.reflect, org.apache.logging.log4j.core.config.xml, com.navercorp.pinpoint.profiler.context.provider.plugin, com.navercorp.pinpoint.grpc.server.flowcontrol, org.apache.thrift.protocol, com.navercorp.pinpoint.profiler.context.recorder.proxy, org.jboss.netty.handler.codec, org.yaml.snakeyaml.extensions.compactnotation, io.netty.handler.codec.marshalling, org.yaml.snakeyaml.external.com.google.gdata.util.common.base, com.navercorp.pinpoint.profiler.monitor.metric.transaction, com.google.errorprone.annotations.concurrent, org.apache.logging.log4j.core.config.plugins, io.netty.handler.ssl.util, com.navercorp.pinpoint.profiler.context.grpc.config, com.navercorp.pinpoint.profiler.interceptor.factory, io.grpc.inprocess, com.navercorp.pinpoint.profiler.monitor.collector.totalthread, com.google.logging.type, org.apache.logging.log4j.core.config.properties, org.slf4j.spi, org.jboss.netty.container.spring, io.netty.util.internal.shaded.org.jctools.queues.atomic, com.navercorp.pinpoint.grpc.security, com.navercorp.pinpoint.grpc.channelz, org.jboss.netty.handler.codec.http.cookie, com.navercorp.pinpoint.common.profiler.trace, org.jboss.netty.handler.traffic, org.apache.thrift.transport, com.navercorp.pinpoint.common.profiler.util, org.apache.logging.log4j.core.filter, com.navercorp.pinpoint.profiler.receiver.grpc, com.google.common.xml, com.navercorp.pinpoint.loader.plugins.trace.yaml, com.navercorp.pinpoint.profiler.sender, com.navercorp.pinpoint.profiler.monitor.metric.response, io.netty.util.internal.shaded.org.jctools.queues, com.navercorp.pinpoint.profiler.context.monitor.config, com.navercorp.pinpoint.profiler.context.errorhandler, com.navercorp.pinpoint.agent.plugin.proxy.common, io.netty.handler.codec.base64, com.navercorp.pinpoint.profiler.context.provider, org.jboss.netty.handler.codec.http.websocketx, com.navercorp.pinpoint.profiler.monitor.metric.totalthread, com.navercorp.pinpoint.grpc.server, com.google.inject.internal.cglib.transform, com.google.protobuf.compiler, org.apache.logging.log4j.message, com.navercorp.pinpoint.loader.service, com.google.j2objc.annotations, org.jboss.netty.logging, io.netty.util.collection, com.google.common.graph, org.jboss.netty.handler.codec.frame, io.netty.channel.epoll, org.objectweb.asm.commons, io.netty.handler.codec.http.websocketx.extensions, io.netty.resolver, org.apache.logging.log4j.core.parser, org.apache.logging.log4j.status, com.navercorp.pinpoint.profiler.receiver, org.apache.logging.log4j.core.config, io.netty.handler.codec.http2, org.apache.thrift.annotation, com.navercorp.pinpoint.profiler.context.provider.metadata, com.navercorp.pinpoint.profiler.interceptor.registry, com.navercorp.pinpoint.profiler.monitor.metric.loadedclass, io.netty.handler.timeout, io.netty.handler.codec.bytes, org.yaml.snakeyaml.util, org.yaml.snakeyaml.resolver, io.netty.channel.oio, org.apache.logging.log4j.core.config.plugins.util, org.jboss.netty.handler.codec.marshalling, com.navercorp.pinpoint.profiler.context.provider.stat.datasource, com.google.common.hash, com.navercorp.pinpoint.profiler.context.provider.stat.transaction, org.apache.logging.log4j.core.lookup, io.netty.handler.ssl.ocsp, com.navercorp.pinpoint.common.profiler.concurrent, org.apache.logging.log4j.core.util.datetime, com.navercorp.pinpoint.loader.plugins.trace, org.yaml.snakeyaml.representer, com.navercorp.pinpoint.profiler.monitor.collector.buffer, com.navercorp.pinpoint.profiler.util.jdk, com.navercorp.pinpoint.rpc.packet, org.yaml.snakeyaml.reader, com.navercorp.pinpoint.profiler.monitor.metric.custom, com.navercorp.pinpoint.profiler.monitor.collector.transaction, com.navercorp.pinpoint.profiler.context.provider.grpc, com.navercorp.pinpoint.profiler.sender.grpc.stream, io.netty.util.internal.logging, org.apache.logging.log4j.core.time.internal, com.navercorp.pinpoint.profiler.instrument.config, io.netty.channel.nio, org.checkerframework.checker.nullness.compatqual, org.slf4j.impl, org.jboss.netty.handler.codec.http.multipart, com.navercorp.pinpoint.grpc.client, io.grpc.protobuf, com.navercorp.pinpoint.profiler.instrument.transformer, io.netty.handler.codec.compression, com.navercorp.pinpoint.profiler.monitor, org.aopalliance.intercept, com.navercorp.pinpoint.profiler.context.provider.stat.response, io.netty.channel.pool, com.google.inject.spi, org.apache.logging.log4j.core.config.plugins.visitors, com.navercorp.pinpoint.profiler.monitor.metric.memory, org.apache.logging.log4j.core.appender.nosql, com.google.gson.reflect, org.apache.logging.log4j.core.appender.db.jdbc, org.apache.logging.log4j.core, com.navercorp.pinpoint.io.header.v2, com.navercorp.pinpoint.profiler.monitor.metric.filedescriptor, com.navercorp.pinpoint.io.header.v1, com.google.inject.internal.asm, io.netty.util.internal.shaded.org.jctools.util, com.navercorp.pinpoint.profiler.context.thrift, com.google.common.reflect, com.navercorp.pinpoint.common.profiler.sql, org.yaml.snakeyaml.external.biz.base64Coder, com.navercorp.pinpoint.grpc.server.lifecycle, com.navercorp.pinpoint.profiler.logging.jul, com.navercorp.pinpoint.profiler.context.provider.stat.deadlock, org.objectweb.asm.util, io.netty.handler.address, org.yaml.snakeyaml.nodes, com.google.inject.binder, org.apache.thrift.meta_data, org.apache.logging.log4j.core.appender.mom.jeromq, com.navercorp.pinpoint.rpc.stream, com.navercorp.pinpoint.profiler.monitor.collector, org.jboss.netty.handler.stream, org.yaml.snakeyaml.constructor, org.apache.logging.log4j.core.net.ssl, com.navercorp.pinpoint.profiler.receiver.service, org.yaml.snakeyaml.serializer, org.jboss.netty.handler.queue, org.jboss.netty.bootstrap, com.navercorp.pinpoint.profiler.context.scope, com.navercorp.pinpoint.profiler.monitor.metric.datasource, org.jboss.netty.handler.codec.protobuf, com.navercorp.pinpoint.profiler.transformer, com.navercorp.pinpoint.profiler.monitor.metric.cpu, org.yaml.snakeyaml.env, org.jboss.netty.channel.local, org.apache.logging.log4j.core.impl, com.navercorp.pinpoint.common.profiler.encoding, org.yaml.snakeyaml.parser, org.apache.logging.log4j.core.config.json, com.navercorp.pinpoint.profiler.instrument.interceptor, io.netty.bootstrap, com.navercorp.pinpoint.rpc.buffer, org.jboss.netty.handler.codec.http, org.apache.logging.log4j.util, com.navercorp.pinpoint.thrift.dto.flink, io.netty.channel.local, io.perfmark, com.navercorp.pinpoint.profiler.monitor.collector.filedescriptor, com.navercorp.pinpoint.io.header, com.navercorp.pinpoint.profiler.context.monitor, com.navercorp.pinpoint.profiler.monitor.collector.response, com.navercorp.pinpoint.rpc.client, com.google.common.annotations, org.jboss.netty.container.microcontainer, org.jboss.netty.handler.codec.string, io.netty.channel.group, org.apache.logging.log4j.core.jmx, com.navercorp.pinpoint.loader.plugins.profiler, io.grpc.util, org.apache.logging.log4j.core.jackson, org.objectweb.asm.tree, com.navercorp.pinpoint.profiler.instrument.scanner, io.netty.util.internal.svm, org.jboss.netty.handler.logging, com.navercorp.pinpoint.agent.plugin.proxy.nginx, org.jboss.netty.handler.codec.oneone, com.navercorp.pinpoint.profiler.context.provider.stat.activethread, io.netty.internal.tcnative, io.netty.handler.pcap, org.apache.logging.log4j.core.config.plugins.validation, javax.annotation, io.netty.handler.codec.xml, com.google.inject.internal.cglib.proxy, org.apache.logging.log4j.core.time, javax.annotation.concurrent, com.google.geo.type, com.navercorp.pinpoint.profiler.context.monitor.metric, org.apache.logging.log4j.core.appender.db, io.netty.util, com.navercorp.pinpoint.profiler.logging, org.yaml.snakeyaml.scanner, com.navercorp.pinpoint.profiler.sender.grpc.metric, com.navercorp.pinpoint.profiler.monitor.collector.datasource, org.apache.logging.log4j.spi, io.netty.handler.codec.http.websocketx, com.navercorp.pinpoint.profiler.context.thrift.config, org.jboss.netty.handler.codec.base64, org.apache.thrift, org.apache.logging.log4j.core.config.builder.api, com.navercorp.pinpoint.profiler.context.method, org.jboss.netty.util.internal, com.google.inject.internal.cglib.reflect, io.netty.handler.logging, org.apache.logging.log4j.core.config.plugins.processor, org.jboss.netty.handler.codec.embedder, io.netty.handler.codec.http.cors, io.grpc.stub, org.apache.logging.log4j.core.pattern, org.yaml.snakeyaml.events, org.jboss.netty.handler.codec.rtsp, com.navercorp.pinpoint.agent.plugin.proxy.app, org.jboss.netty.handler.codec.socks, io.netty.handler.flow, com.navercorp.pinpoint.profiler.objectfactory, io.grpc.netty, com.navercorp.pinpoint.profiler.sender.grpc, com.google.common.io, com.google.gson.internal.bind.util, org.apache.logging.log4j.simple, org.apache.thrift.server]
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] providers:{io.grpc.ServerProvider=[io.grpc.netty.NettyServerProvider], io.grpc.NameResolverProvider=[io.grpc.internal.DnsNameResolverProvider], io.grpc.LoadBalancerProvider=[io.grpc.internal.PickFirstLoadBalancerProvider, io.grpc.util.SecretRoundRobinLoadBalancerProvider$Provider], io.grpc.ManagedChannelProvider=[io.grpc.netty.NettyChannelProvider], com.navercorp.pinpoint.agent.plugin.proxy.common.ProxyRequestMetadataProvider=[com.navercorp.pinpoint.agent.plugin.proxy.apache.ApacheRequestMetadataProvider, com.navercorp.pinpoint.agent.plugin.proxy.user.UserRequestMetadataProvider, com.navercorp.pinpoint.agent.plugin.proxy.nginx.NginxRequestMetadataProvider, com.navercorp.pinpoint.agent.plugin.proxy.app.AppRequestMetadataProvider], org.apache.logging.log4j.spi.Provider=[org.apache.logging.log4j.core.impl.Log4jProvider], org.apache.logging.log4j.util.PropertySource=[org.apache.logging.log4j.util.SystemPropertiesPropertySource, org.apache.logging.log4j.util.EnvironmentPropertySource], reactor.blockhound.integration.BlockHoundIntegration=[io.netty.util.internal.Hidden$NettyBlockHoundIntegration], com.navercorp.pinpoint.profiler.context.recorder.proxy.ProxyRequestParserProvider=[com.navercorp.pinpoint.agent.plugin.proxy.app.AppRequestParserProvider, com.navercorp.pinpoint.agent.plugin.proxy.user.UserRequestParserProvider, com.navercorp.pinpoint.agent.plugin.proxy.apache.ApacheRequestParserProvider, com.navercorp.pinpoint.agent.plugin.proxy.nginx.NginxRequestParserProvider], javax.annotation.processing.Processor=[org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor], org.apache.logging.log4j.message.ThreadDumpMessage$ThreadInfoFactory=[org.apache.logging.log4j.core.message.ExtendedThreadInfoFactory]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleBuilder] defineModule module:module pinpoint.agent
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] com.navercorp.pinpoint.test package not found
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] loadModule:java.instrument
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] loadModule:java.management
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] loadModule:jdk.management
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] loadModule:jdk.unsupported
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] load provider:Providers{services='io.grpc.NameResolverProvider', providers=[io.grpc.internal.DnsNameResolverProvider]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] discard provider:Providers{services='javax.annotation.processing.Processor', providers=[org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] load provider:Providers{services='com.navercorp.pinpoint.profiler.context.recorder.proxy.ProxyRequestParserProvider', providers=[com.navercorp.pinpoint.agent.plugin.proxy.app.AppRequestParserProvider, com.navercorp.pinpoint.agent.plugin.proxy.user.UserRequestParserProvider, com.navercorp.pinpoint.agent.plugin.proxy.apache.ApacheRequestParserProvider, com.navercorp.pinpoint.agent.plugin.proxy.nginx.NginxRequestParserProvider]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] discard provider:Providers{services='io.grpc.ServerProvider', providers=[io.grpc.netty.NettyServerProvider]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] discard provider:Providers{services='org.apache.logging.log4j.message.ThreadDumpMessage$ThreadInfoFactory', providers=[org.apache.logging.log4j.core.message.ExtendedThreadInfoFactory]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] load provider:Providers{services='io.grpc.ManagedChannelProvider', providers=[io.grpc.netty.NettyChannelProvider]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] discard provider:Providers{services='org.apache.logging.log4j.util.PropertySource', providers=[org.apache.logging.log4j.util.SystemPropertiesPropertySource, org.apache.logging.log4j.util.EnvironmentPropertySource]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] discard provider:Providers{services='org.apache.logging.log4j.spi.Provider', providers=[org.apache.logging.log4j.core.impl.Log4jProvider]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] load provider:Providers{services='com.navercorp.pinpoint.agent.plugin.proxy.common.ProxyRequestMetadataProvider', providers=[com.navercorp.pinpoint.agent.plugin.proxy.apache.ApacheRequestMetadataProvider, com.navercorp.pinpoint.agent.plugin.proxy.user.UserRequestMetadataProvider, com.navercorp.pinpoint.agent.plugin.proxy.nginx.NginxRequestMetadataProvider, com.navercorp.pinpoint.agent.plugin.proxy.app.AppRequestMetadataProvider]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] load provider:Providers{services='io.grpc.LoadBalancerProvider', providers=[io.grpc.internal.PickFirstLoadBalancerProvider, io.grpc.util.SecretRoundRobinLoadBalancerProvider$Provider]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] discard provider:Providers{services='reactor.blockhound.integration.BlockHoundIntegration', providers=[io.netty.util.internal.Hidden$NettyBlockHoundIntegration]}
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] loadModule:java.xml
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] loadModule:java.desktop
[com.navercorp.pinpoint.bootstrap.java9.module.ModuleSupport] loadModule:java.logging
06-20 11:23:18.044 INFO PinpointStarter : pinpoint agent [com.navercorp.pinpoint.profiler.DefaultAgent] starting...
06-20 11:23:18.082 INFO Log4j2LoggingSystem : logPath:file:/opt/agent.pinpoint/profiles/release/log4j2.xml
06-20 11:23:22.049 WARN PinpointStarter : pinpoint start failed.
com.navercorp.pinpoint.bootstrap.BootStrapException: execute fail. Error:null
at com.navercorp.pinpoint.bootstrap.ContextClassLoaderExecuteTemplate.execute(ContextClassLoaderExecuteTemplate.java:51)
at com.navercorp.pinpoint.bootstrap.AgentBootLoader.boot(AgentBootLoader.java:47)
at com.navercorp.pinpoint.bootstrap.PinpointStarter.start(PinpointStarter.java:140)
at com.navercorp.pinpoint.bootstrap.PinpointBootStrap.premain(PinpointBootStrap.java:89)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:560)
at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:572)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
at com.navercorp.pinpoint.bootstrap.AgentBootLoader$1.call(AgentBootLoader.java:52)
at com.navercorp.pinpoint.bootstrap.ContextClassLoaderExecuteTemplate.execute(ContextClassLoaderExecuteTemplate.java:41)
... 7 more
Caused by: java.lang.IllegalAccessError: class io.opentelemetry.javaagent.instrumentation.netty.v4.common.FutureListenerWrappers (in unnamed module @0x49469ffa) cannot access class io.netty.util.concurrent.GenericFutureListener (in module pinpoint.agent) because module pinpoint.agent does not export io.netty.util.concurrent to unnamed module @0x49469ffa
at pinpoint.agent//io.opentelemetry.javaagent.instrumentation.netty.v4.common.FutureListenerWrappers.<clinit>(FutureListenerWrappers.java:32)
at pinpoint.agent/pinpoint.agent/io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:177)
at pinpoint.agent/pinpoint.agent/io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:35)
at pinpoint.agent/pinpoint.agent/io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:123)
at pinpoint.agent/pinpoint.agent/io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:58)
at pinpoint.agent/pinpoint.agent/io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:52)
at pinpoint.agent/pinpoint.agent/io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:116)
at pinpoint.agent/pinpoint.agent/io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:80)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.grpc.client.ChannelTypeFactory$EpollChannelType.newEventLoopGroup(ChannelTypeFactory.java:46)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.grpc.client.DefaultChannelFactory.<init>(DefaultChannelFactory.java:101)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.grpc.client.DefaultChannelFactoryBuilder.build(DefaultChannelFactoryBuilder.java:107)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.profiler.context.provider.grpc.SpanGrpcDataSenderProvider.get(SpanGrpcDataSenderProvider.java:99)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.profiler.context.provider.grpc.SpanGrpcDataSenderProvider.get(SpanGrpcDataSenderProvider.java:55)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:85)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.BoundProviderFactory.provision(BoundProviderFactory.java:77)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:59)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:61)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ExposedKeyFactory.get(ExposedKeyFactory.java:57)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:60)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:211)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:182)
at pinpoint.agent/pinpoint.agent/com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
at pinpoint.agent/pinpoint.agent/com.google.inject.Guice.createInjector(Guice.java:87)
at pinpoint.agent/pinpoint.agent/com.google.inject.Guice.createInjector(Guice.java:78)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.profiler.context.module.DefaultApplicationContext.<init>(DefaultApplicationContext.java:96)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.profiler.DefaultAgent.newApplicationContext(DefaultAgent.java:111)
at pinpoint.agent/pinpoint.agent/com.navercorp.pinpoint.profiler.DefaultAgent.<init>(DefaultAgent.java:87)
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
... 11 more
*****************************************************************************
* Pinpoint Agent load failure
*****************************************************************************
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2024-06-20 11:23:22 [main] INFO init.Banner -
Additional context
script.sh (Build Script)
#!/bin/bash
# JDK 21 system property setting
export JAVA_HOME=/path/to/java-21/jdk-21.0.2
export PATH=$JAVA_HOME/bin:$PATH
### Log
CURRENT_TIME="$(date "+%Y-%m-%d_%H-%M-%S")"
LOG_FILE="/path/to/logs/server2-${CURRENT_TIME}.log"
### Otel
OTEL_AGENT_PATH="/opt/Path..
OTEL_CONFIG_PATH="/opt/Path.."
### Pinpoint
PINPOINT_AGENT_PATH="/path/to/pinpoint-agent/pinpoint-bootstrap-2.3.3.jar"
PINPOINT_AGENT_ID="pinpoint_agent"
PINPOINT_APPLICATION_NAME="collector_java"
PINPOINT_CONFIG="/path/to/pinpoint-config/pinpoint-root.config"
# nohup java -Dconfig=root-config-2.txt -jar collector_java.jar > ${LOG_FILE} 2>&1 &
nohup java \
-javaagent:$OTEL_AGENT_PATH \
-Dotel.javaagent.configuration-file=$OTEL_CONFIG_PATH \
-Dconfig=root-config.txt \
-javaagent:$PINPOINT_AGENT_PATH \
-Dpinpoint.agentId=$PINPOINT_AGENT_ID \
-Dpinpoint.applicationName=$PINPOINT_APPLICATION_NAME \
-Dpinpoint.config=$PINPOINT_CONFIG \
-jar collector_java.jar > ${LOG_FILE} 2>&1 &