datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

Expose CopyExec metrics in Spark

Open andygrove opened this issue 1 year ago • 0 comments

What is the problem the feature request solves?

In the native planner, we insert a CopyExec node into the plan. The timings from this operator are not visible in the Spark UI.

Describe the potential solution

I think there are two choices:

  1. Create a JVM equivalent CopyExec and make this available in the plan in Spark
  2. Have CopyExec include its time in its child operators metrics

Additional context

No response

andygrove avatar Sep 11 '24 13:09 andygrove