Trace id for sqs messages is not propagated

[aarrsseni]

Describe the bug

I have service which is publishing sqs message to the queue and also is reading the message from this queue. It's using javaagent and have opentelemetry-aws-sdk-2.2-autoconfigure dependency. Parameters described here https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/aws-sdk/README.md are enabled as well.

I see that trace id is not propagated and there is no trace and span id at all after message consuming.

Steps to reproduce

Spring service publishes simple sqs message to the queue using spring-cloud-aws-sqs SqsTemplate. Consume the message using @SqsListener from spring-cloud-aws-sqs. Check the trace id before and after sending.

Expected behavior

Trace id and span id should be propagated and shouldn't be lost.

Actual behavior

Trace id and span id are lost after sending sqs message.

Javaagent or library instrumentation version

2.5.0

Environment

JDK: Temurin 17.0.7 OS: Alpine Linux Spring: 6.1.10 opentelemetry-aws-sdk-2.2-autoconfigure: 2.7.0-alpha spring-cloud-aws-sqs: 3.1.1

Additional context

No response

[laurit]

Since camel provides its own open telemetry integration you should seek assistance from the camel community. If you are able to reproduce this without using camel then please provide a minimal application that reproduces the issue along with any necessary instructions.

[aarrsseni]

@laurit, thanks for the reply about camel. I checked another service I have, which is not using Camel at all. I updated task description with the new case. It's using spring-cloud-aws-sqs, do you know, has open telemetry support for it? Should I configure something else? In any way for now looks like there is not enough just to add one dependency and enable few flags to propagate trace id.

[laurit]

It's using javaagent and have opentelemetry-aws-sdk-2.2-autoconfigure dependency.

When using java agent you don't need to include opentelemetry-aws-sdk-2.2-autoconfigure.

Spring service publishes simple sqs message to the queue using spring-cloud-aws-sqs SqsTemplate. Consume the message using @SqsListener from spring-cloud-aws-sqs. Check the trace id before and after sending.

Please provide minimal application that reproduces the issue along with any instructions necessary.

[aarrsseni]

@laurit, here is an example https://github.com/aarrsseni/demo-sqs-opentelemetry/tree/master/src/main. I checked in my env and I had trace id for rest request but no trace id for sqs. As I mentioned I'm using javaagent 2.5.0. Now I haven't included any opentelemetry dependencies, only javaagent.

[laurit]

@aarrsseni your configuration https://github.com/aarrsseni/demo-sqs-opentelemetry/blob/73a79bbcb059f70ee244fc551d8b2ef642c20eeb/src/main/resources/application.yml#L9 sets endpoint to http://docker:4566 is that localstack? If it is localstack then as far as I know localstack propagates aws trace headers only in commercial version. Idk if that has changed. If that is the case see https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/aws-sdk/aws-sdk-2.2/library/README.md#trace-propagation there is an experimental option to use a different propagation mechanism. If it is not localstack then please elaborate what it is.

[aarrsseni]

@laurit, yes, this is LocalStack in the example but I have the same configuration in the real env without LocalStack. I've checked parameters described in the documentation as well but it didn't help.

Maybe you have an example of working solution to check the config or some more detailed documentation?

[laurit]

This will require instrumentation for @SqsListener. There is also https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/4788

[psneha716]

@laurit I have a similar use case as aarrsseni. Tried using the latest javaagent 2.9 with the following configuration but I see that the trace id is not propagated and I'm not able to see the consumer span under the same trace:

JDK: openjdk:18-jdk-alpine3.15 org.springframework.boot:spring-boot-starter-parent:2.7.0 io.awspring.cloud:spring-cloud-starter-aws:2.3.0 io.awspring.cloud:spring-cloud-starter-aws-messaging:2.3.0

Are the changes in 2.9 javaagent version applicable only for spring boot version>=3?

[laurit]

Are the changes in 2.9 javaagent version applicable only for spring boot version>=3?

I believe so

[tienduong-7]

@laurit I have a similar use case as aarrsseni. Tried using the latest javaagent 2.9 with the following configuration but I see that the trace id is not propagated and I'm not able to see the consumer span under the same trace:

JDK: openjdk:18-jdk-alpine3.15 org.springframework.boot:spring-boot-starter-parent:2.7.0 io.awspring.cloud:spring-cloud-starter-aws:2.3.0 io.awspring.cloud:spring-cloud-starter-aws-messaging:2.3.0

Are the changes in 2.9 javaagent version applicable only for spring boot version>=3?

Hi @psneha716, Did you have a chance to test it? I checked on my side, it's not working for @SqsListener, I even use javaagent Version 2.20.1 but still not working.