spark
spark copied to clipboard
[SPARK-40506]Spark Streaming metrics name doesn't need application name
What changes were proposed in this pull request?
This PR removes application name in spark streaming metrics name.
Spark StreamingSource Metrics sourceName is inappropriate.The label now looks like application_xxxxx_xxxx_driver_NetworkWordCount_StreamingMetrics_streaming_lastCompletedBatch_processingEndTime , instead of application_xxxxx_xxxx_driver_StreamingMetrics_streaming_lastCompletedBatch_processingEndTime, the Spark app name is not need.
This makes it hard to use metrics for different Spark applications over time. And this makes the metrics sourceName standard inconsistent
Why are the changes needed?
Removing the application name in spark streaming metrics name is consistent with the metrics name for the other metrics namespace.
Does this PR introduce any user-facing change?
No
How was this patch tested?
N/A
Can one of the admins verify this patch?
Hi, @HeartSaVioR could you please take a look whenever you have a chance? Thanks!