datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

[wip] feat: Add framework for supporting multiple telemetry providers

Open andygrove opened this issue 7 months ago • 2 comments

Which issue does this PR close?

Part of https://github.com/apache/datafusion-comet/issues/1718

Rationale for this change

Experimenting with supporting OpenTelemetry as an option.

What changes are included in this PR?

How are these changes tested?

With otel enabled, I see these warning in the executor logs:

25/05/08 11:42:59 WARN GrpcExporter: Failed to export spans. Server responded with gRPC status code 2. Error message: Failed to connect to localhost/127.0.0.1:4317
25/05/08 11:43:01 WARN GrpcExporter: Failed to export metrics. Server responded with gRPC status code 2. Error message: Failed to connect to localhost/127.0.0.1:4317

andygrove avatar May 08 '25 00:05 andygrove

Codecov Report

Attention: Patch coverage is 50.00000% with 35 lines in your changes missing coverage. Please review.

Project coverage is 58.64%. Comparing base (f09f8af) to head (c53c433). Report is 183 commits behind head on main.

Files with missing lines Patch % Lines
...ache/comet/telemetry/ChromeTelemetryProvider.scala 0.00% 26 Missing :warning:
...apache/comet/telemetry/OpenTelemetryProvider.scala 0.00% 7 Missing :warning:
...org/apache/comet/telemetry/TelemetryProvider.scala 80.00% 0 Missing and 2 partials :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1722      +/-   ##
============================================
+ Coverage     56.12%   58.64%   +2.51%     
- Complexity      976     1147     +171     
============================================
  Files           119      133      +14     
  Lines         11743    12696     +953     
  Branches       2251     2369     +118     
============================================
+ Hits           6591     7445     +854     
- Misses         4012     4072      +60     
- Partials       1140     1179      +39     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar May 08 '25 02:05 codecov-commenter

Current status:

Chrome

2025-05-09_08-11

OpenTelemetry

I can see data in Jaegar, but I need to propagate the span in order to see nested events.

2025-05-09_08-12

andygrove avatar May 09 '25 14:05 andygrove