spark
spark copied to clipboard
[SPARK-40272][CORE]Support service port custom with range
What changes were proposed in this pull request?
Port range specification is supported by specifying the starting port and configuring the number of port retries.Introduction of a new parameters: spark.service.port.custom.origin
Service like SparkUI.port, SparkDriver.port, NettyService.port etc. The starting port number is spark.service.port.custom.origin and the port range is limited by spark.port.maxRetries. the final port range is: spark.service.port.custom.origin <= port.range <= spark.service.port.custom.origin + spark.port.maxRetries
e.g. spark.service.port.custom.origin=49152 and spark.port.maxRetries=10, the actual range of the port is greater than or equal to 49152 and less than 49163
Why are the changes needed?
In a real production environment, there are some special scenarios where we need to limit the scope of the service port.
Does this PR introduce any user-facing change?
No
How was this patch tested?
new ut
Can one of the admins verify this patch?
@HyukjinKwon Hi,would you mind to take another look?
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!