greys-anatomy icon indicating copy to clipboard operation
greys-anatomy copied to clipboard

Elasticsearch 为什么不能观察呢

Open whitelilis opened this issue 7 years ago • 9 comments

image

如上图,不明觉厉

另外,确认 3658 端口没有占用

whitelilis avatar Jan 12 '18 02:01 whitelilis

找到程序的 out 日志中有如下东东

java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "createClassLoader")
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
        at java.security.AccessController.checkPermission(AccessController.java:884)
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
        at java.lang.SecurityManager.checkCreateClassLoader(SecurityManager.java:611)
        at java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:274)
        at java.lang.ClassLoader.<init>(ClassLoader.java:335)
        at java.security.SecureClassLoader.<init>(SecureClassLoader.java:99)
        at java.net.URLClassLoader.<init>(URLClassLoader.java:144)
        at com.github.ompc.greys.agent.AgentClassLoader.<init>(AgentClassLoader.java:15)
        at com.github.ompc.greys.agent.AgentLauncher.loadOrDefineClassLoader(AgentLauncher.java:43)
        at com.github.ompc.greys.agent.AgentLauncher.main(AgentLauncher.java:106)
        at com.github.ompc.greys.agent.AgentLauncher.agentmain(AgentLauncher.java:20)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
        at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:411)

whitelilis avatar Jan 12 '18 06:01 whitelilis

在它的 policy 文件中打开各种限制

permission java.lang.RuntimePermission "createClassLoader";
  permission java.lang.RuntimePermission "shutdownHooks";
  permission java.io.FilePermission "~/.greys/lib/1.7.6.6/greys/greys-core.jar", "read";

报下面的错:

java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.github.ompc.greys.agent.AgentLauncher.main(AgentLauncher.java:131)
        at com.github.ompc.greys.agent.AgentLauncher.agentmain(AgentLauncher.java:20)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
        at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:411)
Caused by: java.lang.IllegalArgumentException: hostname can't be null
        at java.net.InetSocketAddress.checkHost(InetSocketAddress.java:149)
        at java.net.InetSocketAddress.<init>(InetSocketAddress.java:216)
        at com.github.ompc.greys.core.server.GaServer.getInetSocketAddress(GaServer.java:223)
        at com.github.ompc.greys.core.server.GaServer.bind(GaServer.java:200)
        ... 12 more

whitelilis avatar Jan 12 '18 07:01 whitelilis

可以排除是机器配置或环境的问题,因为 greys 其它 java 进程是可以的。只有 ES 的不行。

whitelilis avatar Jan 12 '18 08:01 whitelilis

Caused by: java.lang.IllegalArgumentException: hostname can't be null
        at java.net.InetSocketAddress.checkHost(InetSocketAddress.java:149)
        at java.net.InetSocketAddress.<init>(InetSocketAddress.java:216)
        at com.github.ompc.greys.core.server.GaServer.getInetSocketAddress(GaServer.java:223)
        at com.github.ompc.greys.core.server.GaServer.bind(GaServer.java:200)
        ... 12 more

问题出在这里,说明greys.sh在attach的时候传递给JVM的参数中,TARGET_IP参数为空...不应该啊,而且只在Elasticsearch中出现这个问题,百思不得其解。

能否帮个小忙,在greys.sh中debug下,看在这个脚本中的attach_jvm方法相关的参数到底是怎样的?

# attach greys to target jvm
# $1 : greys_local_version
attach_jvm()
{
    local greys_lib_dir=${GREYS_LIB_DIR}/${1}/greys

    # if [ ${TARGET_IP} = ${DEFAULT_TARGET_IP} ]; then
    if [ ! -z ${TARGET_PID} ]; then
        ${JAVA_HOME}/bin/java \
            ${BOOT_CLASSPATH} ${JVM_OPTS} \
            -jar ${greys_lib_dir}/greys-core.jar \
                -pid ${TARGET_PID} \
                -target ${TARGET_IP}":"${TARGET_PORT} \
                -core "${greys_lib_dir}/greys-core.jar" \
                -agent "${greys_lib_dir}/greys-agent.jar"
    fi
}

oldmanpushcart avatar Jan 12 '18 12:01 oldmanpushcart

在主日志中也帮忙找下这条日志的输出(如果你的Elasticsearch默认开了INFO级的日志的话),应该在Elasticsearch的主日志中能找到。

/**
 * GaServer操作的附件
 * Created by [email protected] on 15/5/3.
 */
class GaAttachment {

    // ...以下省略千言万语

    /**
     * 启动Greys服务端
     *
     * @param configure 配置信息
     * @throws IOException 服务器启动失败
     */
    public void bind(Configure configure) throws IOException {
       
            // ...以下省略千言万语

            logger.info("ga-server listening on network={};port={};timeout={};", configure.getTargetIp(),
                    configure.getTargetPort(),
                    configure.getConnectTimeout());

            // ...以下省略千言万语

    }

    // ...以下省略千言万语

}

我想看networkport这两个参数的值

oldmanpushcart avatar Jan 12 '18 12:01 oldmanpushcart

醉了。。。。。今天再测试,居然可以用了。。。。 不过,周五的时候我确实远程 debug 了一下,看到 getTargetIp 得到的是 null。 后面我再多测试几次,看能不能复现。

搞错了,原来是另一个同学开了 greys 观察另一个 jvm, 我就连进去了。。。。。。再一次说明了这种不按进程号,按端口号的弊端,哈哈。

image 执行到这一行会出异常,所以下面那 个日志没有打。 出的异常是 image

args 的值如下:

;;targetIp=127.0.0.1;greysCore=/home/liuzhe/.greys/lib/1.7.6.6/greys/greys-core.jar;greysAgent=/home/liuzhe/.greys/lib/1.7.6.6/greys/greys-agent.jar;connectTimeout=6000;targetPort=9999;javaPid=27190;

whitelilis avatar Jan 14 '18 03:01 whitelilis

另外, greys.sh 里执行的参数如下: -target 127.0.0.1:9999

whitelilis avatar Jan 15 '18 09:01 whitelilis

image

whitelilis avatar Jan 15 '18 09:01 whitelilis

收到...我擦,为啥Configure.toString()竟然也被AccessControl拦住了...百撕不得骑姐

oldmanpushcart avatar Jan 16 '18 05:01 oldmanpushcart