spring-cloud-task pod with opentelemetry collector as sidecar does not stop

[zargarf]

trafficstars

Hi

I have a spring-cloud task application that is triggered as expected by a TaskLaunchRequest via spring-cloud-stream. The pod is dynamically created on kubernetes and, once the task has completed, the task correctly completes or errors.

However, I want to add tracing to the task via spring-cloud-sleuth. I can get an opentelemetry collector running as a sidecar when the task is spawned - in order to send my tracing spans. However when I do this, the pod no longer registers as complete or errored. It just keeps on in the running state when the task status has updated correctly. I am using the spring.cloud.task.closecontext_enabled: true property.

The sidecar is implemented using the sidecar injection mechanism of the opentelemetry operator (https://github.com/open-telemetry/opentelemetry-operator) and registered via the use of a label in the deployment properties added to the TaskLaunchRequest "sidecar.opentelemetry.io/inject true".

Is there anyway to get the task to correctly register if a sidecar is used?

Thanks

[cppwfs]

Usually when running ephemeral apps I'd consider using metrics like timer or long timer provided by micrometer vs using tracing for observations.

But as for shutting down a task that may have a thread still running in it can be can be done by setting spring.cloud.task.closecontextEnabled=true to close the context of the application after the task is complete. More can be read about it here: https://docs.spring.io/spring-cloud-task/docs/current/reference/html/#features-lifecycle

[zargarf]

Hi - I'm already using spring.cloud.task.closecontextEnabled=true. but it looks like the sidecar keeps running even though the task pod completes

[cppwfs]

Hello @zargarf , So let me get a bit more information here. If you run a task on your local system when it is implementing Spring Cloud Sleuth, does the task complete normally?

[zargarf]

Hi @cppwfs - we're running on Openshift. If there is no sidecar injected into the task pod that is spawned, the pod competes fine. If there is a sidecar present in the spawned pod (in this case an opentelemetry collector container), then the task container in that pod completes, but the sidecar in the podremains and hence the pod stays running overall

[cppwfs]

If you do not set the spring.cloud.task.closecontextEnabled=true does the task app remaining running after the task has completed?

[cppwfs]

Also what is your setup? are you using spirng-cloud-sleuth-otel? Have you tested this with the latest GA?

[zargarf]

If I leave out spring.cloud.task.closecontextEnabled=true or set it to false, then the task app container remains running as well as the sidecar. If spring.cloud.task.closecontextEnabled is set to true then the task container stops but the sidecar remains - hence the pod is still shown as running

Setup-wise I'm using spring-cloud-starter-sleuth and spring-cloud-sleuth-zipkin - latest versions

[cppwfs]

So in this case the sidecar is waiting for a signal from the Boot/Task app to know when to shutdown?

[cppwfs]

Closing this issue for inactivity. If any additional details are available please add a comment. Thank you!