Python auto-instrumentation: handle musl based containers

[xrmx]

Description: <Describe what has changed.>

Build and and inject musl based python auto-instrumentation if proper annotation is configured:

instrumentation.opentelemetry.io/otel-python-platform: "musl"

This takes a different approach that the stale PR at #2266:

• does not change directory where python sdk is installed in the docker container for default distribution (glibc)

Link to tracking Issue(s):

• Resolves: #2264

Testing: unit tests and e2e are green. Tested locally on minikube that by deploying an operator from this branch, a custom image for the auto-instrumentation was able to get metrics (that depends on psutil that uses a binary extension) out of a python alpine container with the instrumentation.opentelemetry.io/otel-python-platform: "musl" annotation and a stacktrace without it. Also tested that the glibc based is copied to the container if the instrumentation image does not have the musl one.

Documentation: <Describe the documentation added.> Will add docs to opentelemetry.io

[xrmx]

AFAICS e2e tests are failing because the python autoinstrumentation docker image does not have the musl based installation, is it the case because the image used in tests is not built from git?

[swiatekm]

AFAICS e2e tests are failing because the python autoinstrumentation docker image does not have the musl based installation, is it the case because the image used in tests is not built from git?

Autoinstrumentation tests use the default image, we don't build all the autoinstrumentation images from source for E2E tests, though we probably should.

What I would suggest here:

1. Verify locally that your current tests pass with the new image
2. Modify tests in this PR to only check if the container manifest is correct (and not that it's running)
3. Merge this PR
4. Operator release happens
5. Open a new PR, changing the tests back to their current state

I know it's a hassle, but it's simpler than all the alternatives.

[xrmx]

AFAICS e2e tests are failing because the python autoinstrumentation docker image does not have the musl based installation, is it the case because the image used in tests is not built from git?

Autoinstrumentation tests use the default image, we don't build all the autoinstrumentation images from source for E2E tests, though we probably should.

What I would suggest here:

1. Verify locally that your current tests pass with the new image

2. Modify tests in this PR to only check if the container manifest is correct (and not that it's running)

3. Merge this PR

4. Operator release happens

5. Open a new PR, changing the tests back to their current state

I know it's a hassle, but it's simpler than all the alternatives.

Other than fixing the tests though this is breaking backward compatibility with older images and maybe we can avoid that?

[xrmx]

Ok finally tested this manually by deploying an operator from this branch, a custom image for the auto-instrumentation and was able to get metrics (that depends on psutil that uses a binary extension) out of a python alpine container with the instrumentation.opentelemetry.io/otel-python-platform: "musl" annotation and a stacktrace without it.

Also tested that the glibc based is copied to the container if the instrumentation image does not have the musl one.

[xrmx]

I'm not an approver here but this looks great @xrmx. Do we have a place where we can document any of this for users of this functionality?

Thanks for reviewing, will update https://opentelemetry.io/docs/kubernetes/operator/automatic/ and https://opentelemetry.io/docs/zero-code/python/operator/

[xrmx]

Missing changelog.

It's the first hunk in the diff

[xrmx]

The changes look good to me, thanks for being patient with our feedback on this PR! One small thing I think is still missing is documenting this new annotation in the README here: https://github.com/open-telemetry/opentelemetry-operator?tab=readme-ov-file#opentelemetry-auto-instrumentation-injection.

Updated README and rebased, thanks!

I think you have been more patient with me than the other way around :sweat_smile: