OpenTelemetry #2

[yashgorana]

Description

OpenTelemetry

Syft Backend

• [x] Update OTel packages
• [x] Add OTel auto-instrumentation (programmatic instead of opentelemetry-instrument root proc)
  • [x] fastapi
  • [x] pymongo
  • [x] botocore
• [x] Fix OTel exporter (Jaeger --> OTLP)
• [x] Fix OTel Spans (remove @instrument from classes, @service_methods are always instrumented)
• [ ] Custom workers (in-progress)
  • [ ] Plumb OTEL_SERVICE_NAME, OTEL_EXPORTER_* env vars to worker pool launch
• [ ] ZMQ tracing (in-progress)
  • [ ] instrument zmq messages except heartbeat
  • [ ] zmq messages need to include OTLP span for it to be picked up by the consumer (like Traceparent header)

Syft Client

• [ ] Add OTel span to Syft Client (todo. a trace should capture client -> backend ->worker -> db)
  • [ ] Include requests auto-instrumenter
  • [ ] Client needs an API endpoint to export client traces to backend.
• [ ] Backend uses syft client, in this case we need to export directly to internal provider (???)

Infra

• [x] Dev - UI (tox -re dev.k8s.install.signoz)
• [ ] OTel operator

Affected Dependencies

List any dependencies that are required for this change.

How has this been tested?

• Describe the tests that you ran to verify your changes.
• Provide instructions so we can reproduce.
• List any relevant details for your test configuration.

Checklist

• [x] I have followed the Contribution Guidelines and Code of Conduct
• [x] I have commented my code following the OpenMined Styleguide
• [x] I have labeled this PR with the relevant Type labels
• [ ] My changes are covered by tests