[Feature] add RocketMq consumer mq.topic tag

[zhyyu]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

RocketMQ consumer segment don't contain mq.topic tag, so the UI trace page can not search consumer segment when input mq.topic=TopicTest tag. I want to change RocketMQ agent, add mq.topic tag into consumer segment.

Use case

I checked the agent code and found consumer interceptor AbstractMessageConsumeInterceptor

@Override
    public final void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments,
        Class<?>[] argumentsTypes, MethodInterceptResult result) throws Throwable {
        List<MessageExt> msgs = (List<MessageExt>) allArguments[0];

        ContextCarrier contextCarrier = getContextCarrierFromMessage(msgs.get(0));
        AbstractSpan span = ContextManager.createEntrySpan(CONSUMER_OPERATION_NAME_PREFIX + msgs.get(0)
                                                                                                .getTopic() + "/Consumer", contextCarrier);

        span.setComponent(ComponentsDefine.ROCKET_MQ_CONSUMER);
        SpanLayer.asMQ(span);
        for (int i = 1; i < msgs.size(); i++) {
            ContextManager.extract(getContextCarrierFromMessage(msgs.get(i)));
        }

    }

I add one line Tags.MQ_TOPIC.set(span, msgs.get(0).getTopic()); after span created. I tested it on my local machine, it worked.

Related issues

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[wu-sheng]

We are not accepting more RocketMQ plugin codes, unless plugin test is added with it.

[wu-sheng]

This plugin can't be verified automatically currently, so, you need to follow plugin test document to add tests for supported versions. Then we could continue on new feature.

[zhyyu]

Can you tell me where is the "plugin test document" ? I read java-agent doc and not find it.

[wu-sheng]

https://skywalking.apache.org/docs/skywalking-java/latest/en/setup/service-agent/java-agent/java-plugin-development-guide/#contribute-plugins-to-the-apache-skywalking-repository step 5.

[wu-sheng]

@zhyyu Any update about this?

[zhyyu]

@zhyyu Any update about this?

my mac m1 can not run the bash ./test/plugin/run.sh -f ${scenario_name}

bash ./test/plugin/run.sh -f ${scenario_name}

[INFO] SkyWalking Tomcat Runner Image ..................... FAILURE [  0.559 s]
[INFO] SkyWalking JVM Runner Image ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  18.494 s
[INFO] Finished at: 2021-12-09T10:09:36+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.33.0:build (build) on project tomcat-container: Execution build of goal io.fabric8:docker-maven-plugin:0.33.0:build failed: An API incompatibility was encountered while executing io.fabric8:docker-maven-plugin:0.33.0:build: java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
[ERROR] -----------------------------------------------------
[ERROR] realm =    plugin>io.fabric8:docker-maven-plugin:0.33.0
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy

find https://github.com/fabric8io/docker-maven-plugin/issues/1257 docker-maven-plugin conflict with m1 ship. So I am blocked now...

[wu-sheng]

OK, I don't have a M1 chip for now. Could you try it on any Linux? I think it should work.

[zhyyu]

OK, I don't have a M1 chip for now. Could you try it on any Linux? I think it should work.

I have a x86 windows. I think I can run the script on virtual box on that windows. But it will take some times...

[wu-sheng]

I can see fabric8io/docker-maven-plugin has 0.38.0 released recently, could someone try it? If you have a M1?

[zhyyu]

I can see fabric8io/docker-maven-plugin has 0.38.0 released recently, could someone try it? If you have a M1?

I will try it recently.

[zhyyu]

fabric8io/docker-maven-plugin 0.38.1 worked, but still error

[ERROR] Failed to execute goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.6.1:compile (grpc-build) on project mock-collector: Unable to resolve artifact: Missing:
[ERROR] ----------
[ERROR] 1) com.google.protobuf:protoc:exe:osx-aarch_64:3.3.0

I add -Dos.detected.classifier=osx-x86_64 but still don't work.

./test/plugin/run.sh: line 187:  9289 Killed: 9               ${mvnw} --batch-mode -f ${home}/pom.xml -DskipTests -Dos.detected.classifier=osx-x86_64 -Dbase_image_java=${base_image_java} -Dbase_image_tomcat=${base_image_tomcat} -Dcontainer_image_version=${container_image_version} clean package

[lujiajing1126]

fabric8io/docker-maven-plugin 0.38.1 worked, but still error

[ERROR] Failed to execute goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.6.1:compile (grpc-build) on project mock-collector: Unable to resolve artifact: Missing:
[ERROR] ----------
[ERROR] 1) com.google.protobuf:protoc:exe:osx-aarch_64:3.3.0

I add -Dos.detected.classifier=osx-x86_64 but still don't work.

./test/plugin/run.sh: line 187:  9289 Killed: 9               ${mvnw} --batch-mode -f ${home}/pom.xml -DskipTests -Dos.detected.classifier=osx-x86_64 -Dbase_image_java=${base_image_java} -Dbase_image_tomcat=${base_image_tomcat} -Dcontainer_image_version=${container_image_version} clean package

I've checked the issue in Skywalking-java repo. You have to update the protoc which may be hardcoded in many test cases now.

protobuf for Apple Silicon have been supported from 3.17.3, see the comment and related PR there https://github.com/apache/skywalking/issues/8522#issuecomment-1033404584

Also, on M1 models, I was using Ecplise/Temurin distribution which can be used to build the project since it is built for x86_64.

[zhyyu]

Can configuration.yml dependencies support command command, the official rocketmq docker image have to to use this comand like below

https://github.com/apache/rocketmq-docker

version: '2'
services:
  namesrv:
    image: apache/rocketmq:4.5.0
    ...
    command: sh mqnamesrv
  broker:
    image: apache/rocketmq:4.5.0
    ...
    command: sh mqbroker -n namesrv:9876 -c ../conf/broker.conf
    depends_on:
      - namesrv

Right now it don't support command command

Caused by: org.yaml.snakeyaml.error.YAMLException: Unable to find property 'command' on class: org.apache.skywalking.plugin.test.helper.vo.DependencyComponent
        at org.yaml.snakeyaml.introspector.PropertyUtils.getProperty(PropertyUtils.java:158)
        at org.yaml.snakeyaml.introspector.PropertyUtils.getProperty(PropertyUtils.java:148)
        at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.getProperty(Constructor.java:287)
        at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:208)
        ... 20 more

[wu-sheng]

You could add this through a pull request I think. The docker configuration is just generated through the codes inside test folder.

[zhyyu]

You could add this through a pull request I think. The docker configuration is just generated through the codes inside test folder.

Ok, I am trying work on it.

[wu-sheng]

As you are going to follow this, @zhyyu Notice there is an enhancement at the backend to provide MQ relative analysis from spans. Please check this https://github.com/apache/skywalking/pull/9855 and be aware of which tags are required/recommended for the analysis.

[zhyyu]

As you are going to follow this, @zhyyu Notice there is an enhancement at the backend to provide MQ relative analysis from spans. Please check this #9855 and be aware of which tags are required/recommended for the analysis.

Yes, I am going to finish it. Right now the rocketmq-scenario is ready.