arthas icon indicating copy to clipboard operation
arthas copied to clipboard

执行stack命令报错java.lang.UnsupportedOperationException: class redefinition failed: attempted to change superclass or interfaces

Open ruanyg opened this issue 2 years ago • 5 comments

  • [x] 我已经在 issues 里搜索,没有重复的issue。

环境信息

  • arthas-boot.jar 或者 as.sh 的版本: xxx
  • Arthas 版本: 3.6.2
  • 操作系统版本: Linux ubuntu 4.9.0-141-custom
  • 目标进程的JVM版本: 1.8.0_171-b11
  • 执行arthas-boot的版本: xxx

重现问题的步骤

  1. 使用stack命令
  2. 报错 java.lang.UnsupportedOperationException: class redefinition failed: attempted to change superclass or interfaces

期望的结果

期望能够正常执行stack命令

实际运行的结果

2022-08-09 19:36:56 [arthas-command-execute] INFO  c.t.a.core.util.InstrumentationUtils -ignore lambda class: org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask$$Lambda$1487/899812517, because jdk do not support retransform lambda class: https://github.com/alibaba/arthas/issues/1512.
2022-08-09 19:36:56 [arthas-command-execute] INFO  c.t.a.core.util.InstrumentationUtils -ignore lambda class: org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask$$Lambda$1329/682502598, because jdk do not support retransform lambda class: https://github.com/alibaba/arthas/issues/1512.
2022-08-09 19:36:56 [arthas-command-execute] ERROR c.t.a.core.util.InstrumentationUtils -retransformClasses class error, name: org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask
java.lang.UnsupportedOperationException: class redefinition failed: attempted to change superclass or interfaces
	at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
	at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)
	at com.taobao.arthas.core.util.InstrumentationUtils.retransformClasses(InstrumentationUtils.java:32)
	at com.taobao.arthas.core.command.klass100.JadCommand.processExactMatch(JadCommand.java:177)
	at com.taobao.arthas.core.command.klass100.JadCommand.process(JadCommand.java:158)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
	at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:385)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

ruanyg avatar Aug 09 '22 11:08 ruanyg

https://github.com/alibaba/arthas/search?q=attempted+to+change+superclass+or+interfaces&type=issues

hengyunabc avatar Aug 10 '22 07:08 hengyunabc

https://github.com/alibaba/arthas/search?q=attempted+to+change+superclass+or+interfaces&type=issues

jad之后的结果为

/*
         * Decompiled with CFR.
         *
         * Could not load the following classes:
         *  lombok.Generated
         */
        package org.apache.shardingsphere.proxy.frontend.command;

        import io.netty.buffer.ByteBuf;
        import io.netty.channel.ChannelHandlerContext;
        import java.sql.SQLException;
        import java.util.Collection;
        import java.util.LinkedList;
        import java.util.Optional;
        import lombok.Generated;
        import org.apache.commons.lang3.StringUtils;
        import org.apache.shardingsphere.db.protocol.CommonConstants;
        import org.apache.shardingsphere.db.protocol.packet.CommandPacket;
        import org.apache.shardingsphere.db.protocol.packet.CommandPacketType;
        import org.apache.shardingsphere.db.protocol.packet.DatabasePacket;
        import org.apache.shardingsphere.db.protocol.payload.PacketPayload;
        import org.apache.shardingsphere.proxy.backend.communication.SQLStatementSchemaHolder;
        import org.apache.shardingsphere.proxy.backend.communication.jdbc.connection.BackendConnection;
        import org.apache.shardingsphere.proxy.backend.communication.jdbc.connection.ConnectionStatus;
        import org.apache.shardingsphere.proxy.backend.context.ProxyContext;
        import org.apache.shardingsphere.proxy.backend.text.distsql.ral.common.hint.HintManagerHolder;
        import org.apache.shardingsphere.proxy.frontend.command.CommandExecuteEngine;
        import org.apache.shardingsphere.proxy.frontend.command.executor.CommandExecutor;
        import org.apache.shardingsphere.proxy.frontend.command.executor.QueryCommandExecutor;
        import org.apache.shardingsphere.proxy.frontend.exception.ExpectedExceptions;
        import org.apache.shardingsphere.proxy.frontend.spi.DatabaseProtocolFrontendEngine;
        import org.apache.shardingsphere.transaction.TransactionHolder;
        import org.slf4j.Logger;
        import org.slf4j.LoggerFactory;
        import org.slf4j.MDC;

        public final class CommandExecutorTask
        implements Runnable {
            @Generated
            private static final Logger log = LoggerFactory.getLogger(CommandExecutorTask.class);
            private final DatabaseProtocolFrontendEngine databaseProtocolFrontendEngine;
            private final BackendConnection backendConnection;
            private final ChannelHandlerContext context;
            private final Object message;

            /*
             * WARNING - Removed try catching itself - possible behaviour change.
             */
            @Override
            public void run() {
/* 71*/         boolean isNeedFlush = false;
/* 72*/         try (PacketPayload payload = this.databaseProtocolFrontendEngine.getCodecEngine().createPacketPayload((ByteBuf)this.message, this.context.channel().attr(CommonConstants.CHARSET_ATTRIBUTE_KEY).get());){
                    String availabilityZone;
/* 73*/             this.fillLogMDC();
/* 74*/             if (this.backendConnection.getTransactionStatus().isInTransaction()) {
/* 75*/                 TransactionHolder.setInTransaction();
                    } else {
/* 77*/                 TransactionHolder.clear();
                    }
/* 79*/             ConnectionStatus connectionStatus = this.backendConnection.getConnectionStatus();
/* 80*/             if (!this.backendConnection.getTransactionStatus().isInConnectionHeldTransaction()) {
/* 81*/                 connectionStatus.waitUntilConnectionRelease();
/* 82*/                 connectionStatus.switchToUsing();
                    }
/* 84*/             if (!this.backendConnection.isRws()) {
/* 85*/                 HintManagerHolder.get().setWriteRouteOnly();
                    }
/* 87*/             if (StringUtils.isNotBlank(availabilityZone = ProxyContext.getInstance().getAvailabilityZone())) {
/* 89*/                 HintManagerHolder.get().setAvailabilityZone(availabilityZone);
                    }
/* 91*/             isNeedFlush = this.executeCommand(this.context, payload, this.backendConnection);
                }
                catch (Exception ex) {
/* 95*/             this.processException(ex);
                }
                finally {
/* 98*/             SQLStatementSchemaHolder.remove();
/* 99*/             Collection<SQLException> exceptions = this.closeExecutionResources();
/*100*/             if (isNeedFlush) {
/*101*/                 this.context.flush();
                    }
/*103*/             if (!this.backendConnection.getTransactionStatus().isInConnectionHeldTransaction()) {
/*104*/                 exceptions.addAll(this.backendConnection.closeDatabaseCommunicationEngines(true));
/*105*/                 exceptions.addAll(this.backendConnection.closeConnections(false));
/*106*/                 this.backendConnection.getConnectionStatus().switchToReleased();
                    }
/*108*/             this.processClosedExceptions(exceptions);
/*109*/             HintManagerHolder.get().close();
/*110*/             HintManagerHolder.remove();
/*111*/             this.clearLogMDC();
                }
            }

            /*
             * WARNING - Removed try catching itself - possible behaviour change.
             */
            private boolean executeCommand(ChannelHandlerContext context, PacketPayload payload, BackendConnection backendConnection) throws SQLException {
/*116*/         CommandExecuteEngine commandExecuteEngine = this.databaseProtocolFrontendEngine.getCommandExecuteEngine();
/*117*/         CommandPacketType type = commandExecuteEngine.getCommandPacketType(payload);
/*118*/         CommandPacket commandPacket = commandExecuteEngine.getCommandPacket(payload, type, backendConnection);
/*119*/         try (CommandExecutor commandExecutor = commandExecuteEngine.getCommandExecutor(type, commandPacket, backendConnection);){
/*121*/             Collection<DatabasePacket<?>> responsePackets = commandExecutor.execute();
/*122*/             if (responsePackets.isEmpty()) {
/*123*/                 boolean bl = false;
/*123*/                 return bl;
                    }
/*125*/             responsePackets.forEach(context::write);
/*126*/             if (commandExecutor instanceof QueryCommandExecutor) {
/*127*/                 boolean bl = commandExecuteEngine.writeQueryData(context, backendConnection, (QueryCommandExecutor)commandExecutor, responsePackets.size());
/*127*/                 return bl;
                    }
                }
/*132*/         return this.databaseProtocolFrontendEngine.getFrontendContext().isFlushForPerCommandPacket();
            }

            private void processException(Exception cause) {
/*136*/         if (!ExpectedExceptions.isExpected(cause.getClass())) {
/*137*/             log.error("Exception occur: ", cause);
                }
/*139*/         this.context.write(this.databaseProtocolFrontendEngine.getCommandExecuteEngine().getErrorPacket(cause, this.backendConnection));
/*140*/         Optional<DatabasePacket<?>> databasePacket = this.databaseProtocolFrontendEngine.getCommandExecuteEngine().getOtherPacket(this.backendConnection);
/*141*/         databasePacket.ifPresent(this.context::write);
/*142*/         this.context.flush();
            }

            private Collection<SQLException> closeExecutionResources() {
                LinkedList<SQLException> result = new LinkedList<SQLException>();
/*147*/         result.addAll(this.backendConnection.closeDatabaseCommunicationEngines(false));
/*148*/         result.addAll(this.backendConnection.closeFederationExecutor());
/*149*/         return result;
            }

            private void processClosedExceptions(Collection<SQLException> exceptions) {
/*153*/         if (exceptions.isEmpty()) {
/*154*/             return;
                }
                SQLException ex = new SQLException("");
/*157*/         for (SQLException each : exceptions) {
/*158*/             ex.setNextException(each);
                }
/*160*/         this.processException(ex);
            }

            private void fillLogMDC() {
/*164*/         MDC.put("schemaName", this.backendConnection.getSchemaName());
            }

            private void clearLogMDC() {
/*168*/         MDC.clear();
            }

            @Generated
            public CommandExecutorTask(DatabaseProtocolFrontendEngine databaseProtocolFrontendEngine, BackendConnection backendConnection, ChannelHandlerContext context, Object message) {
                this.databaseProtocolFrontendEngine = databaseProtocolFrontendEngine;
                this.backendConnection = backendConnection;
                this.context = context;
                this.message = message;
            }
        }

和github中搜到的issue情况不太一致

ruanyg avatar Aug 10 '22 07:08 ruanyg

1.8.0_171-b11 这个jdk版本太老了,试下本地和线上用同样的jdk版本。 另外先在本地测试下能不能工作。

hengyunabc avatar Aug 10 '22 08:08 hengyunabc

升级为1.8.0_333仍然报相同错误,启动时带了jaeger相关agent启动参数,如果不带该agent启动可以正常使用stack命令 有办法兼容该agent同时使用stack命令么?

arthas以agent方式启动时使用 stack org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask run 可以看到如下报错

[ERROR] 2022-08-19 15:02:22.014 [arthas-command-execute] [o.a.s.a.c.b.listener.LoggingListener] [] [] [SLDB_ENV_IS_UNDEFINED] - Failed to instrument org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask
java.lang.IllegalStateException: Already implemented interface interface org.apache.shardingsphere.agent.api.advice.AdviceTargetObject for class org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask
	at org.apache.shardingsphere.dependencies.net.bytebuddy.dynamic.scaffold.InstrumentedType$Default.validated(InstrumentedType.java:1476)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.dynamic.scaffold.MethodRegistry$Default.prepare(MethodRegistry.java:520)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.dynamic.scaffold.inline.RebaseDynamicTypeBuilder.make(RebaseDynamicTypeBuilder.java:221)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11362)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11298)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11016)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:11690)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:11637)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
	at org.apache.shardingsphere.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11205)
	at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
	at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
	at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
	at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)
	at com.taobao.arthas.core.advisor.Enhancer.enhance(Enhancer.java:446)
	at com.taobao.arthas.core.command.monitor200.EnhancerCommand.enhance(EnhancerCommand.java:162)
	at com.taobao.arthas.core.command.monitor200.EnhancerCommand.process(EnhancerCommand.java:109)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.process(AnnotatedCommandImpl.java:82)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl.access$100(AnnotatedCommandImpl.java:18)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:111)
	at com.taobao.arthas.core.shell.command.impl.AnnotatedCommandImpl$ProcessHandler.handle(AnnotatedCommandImpl.java:108)
	at com.taobao.arthas.core.shell.system.impl.ProcessImpl$CommandProcessTask.run(ProcessImpl.java:385)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

对应的bytebuddy报错的代码为 image

@hengyunabc

ruanyg avatar Aug 19 '22 04:08 ruanyg

@ruanyg 这个可能和 skywalking 的类似,因为 bytebuddy本身不支持多次增强类, skywalking 是增加一个 cache来解决的。可能 shardingsphere 也要类似操作。在issue里找下 skywalking 相关的。

hengyunabc avatar Aug 23 '22 11:08 hengyunabc