s3proxy icon indicating copy to clipboard operation
s3proxy copied to clipboard

Disable Jetty shading to allow OpenTelemetry instrumentation

Open hermandavid opened this issue 1 month ago • 4 comments

S3proxy container currently shades Jetty classes into a different package. This prevents the OpenTelemetry Java agent from matching Jetty types and injecting server instrumentation. Could we stop relocating Jetty (or publish an unshaded variant in container) so OTEL can instrument it?

hermandavid avatar Nov 10 '25 20:11 hermandavid

References #263.

gaul avatar Nov 10 '25 21:11 gaul

Instead of dropping shading entirely, we could keep the current shaded JAR as the main Maven artifact (so existing users and embedders are not affected) and only update the Dockerfile to include building an unshaded version in the container.

If that sounds acceptable, I’m happy to put together a PR with the new Maven profile and Dockerfile changes.

hermandavid avatar Nov 18 '25 13:11 hermandavid

Not shading the Docker artifact is fine in theory but in practice additional build configuration is fragile. I can merge something like this but if it breaks the build some future PR I will remove it.

What are the actual metrics you want, specifically Jetty metrics that might be exported some other way or something similar to #427 or #780?

gaul avatar Nov 18 '25 18:11 gaul

It’s mostly not about the metrics or logs, but about the traces.

Because Jetty is shaded in the image, the OpenTelemetry Java agent cannot recognize it as Jetty and therefore doesn’t apply its server instrumentation. As a result, S3Proxy doesn’t produce proper entry/server spans for incoming HTTP requests, which effectively breaks OpenTelemetry traces at the S3Proxy layer.

With original Jetty, the agent would be able to instrument it correctly. That would allow me to have proper end-to-end traces. For example, to correlate each Azure Blob request made by S3Proxy with the specific upstream S3/API request and the rest of the processing chain, instead of those downstream calls appearing as disconnected spans.

hermandavid avatar Nov 24 '25 09:11 hermandavid