spark
spark copied to clipboard
[WIP] [SPARK-46884] Spark Connect - ExecutePlanRequest new property - job description
What changes were proposed in this pull request?
Add a new optional property to the ExecutePlanRequest object in the Connect gRPC API to allow callers to set the job description from the client app. This would allow callers to set the description to the actual stack trace of the calling application (for example).
Why are the changes needed?
A slight improvement to callers using the Connect API when viewing jobs in the Spark UI.
Does this PR introduce any user-facing change?
Yes, callers can control what is shown in the Spark UI. The default is the current behaviour.
How was this patch tested?
Manually so far, if there is interest in this then I will try and work out (happy to take any advice here!) on how to write tests for this.
Was this patch authored or co-authored using generative AI tooling?
No