spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-40506]Spark Streaming metrics name doesn't need application name

Open beryllw opened this issue 3 years ago • 2 comments

What changes were proposed in this pull request?

This PR removes application name in spark streaming metrics name. Spark StreamingSource Metrics sourceName is inappropriate.The label now looks like application_xxxxx_xxxx_driver_NetworkWordCount_StreamingMetrics_streaming_lastCompletedBatch_processingEndTime , instead of application_xxxxx_xxxx_driver_StreamingMetrics_streaming_lastCompletedBatch_processingEndTime, the Spark app name is not need. This makes it hard to use metrics for different Spark applications over time. And this makes the metrics sourceName standard inconsistent

Why are the changes needed?

Removing the application name in spark streaming metrics name is consistent with the metrics name for the other metrics namespace.

Does this PR introduce any user-facing change?

No

How was this patch tested?

N/A

beryllw avatar Sep 20 '22 12:09 beryllw

Can one of the admins verify this patch?

AmplabJenkins avatar Sep 20 '22 20:09 AmplabJenkins

Hi, @HeartSaVioR could you please take a look whenever you have a chance? Thanks!

beryllw avatar Sep 21 '22 03:09 beryllw