spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-40272][CORE]Support service port custom with range

Open chong0929 opened this issue 3 years ago • 2 comments

What changes were proposed in this pull request?

Port range specification is supported by specifying the starting port and configuring the number of port retries.Introduction of a new parameters: spark.service.port.custom.origin

Service like SparkUI.port, SparkDriver.port, NettyService.port etc. The starting port number is spark.service.port.custom.origin and the port range is limited by spark.port.maxRetries. the final port range is: spark.service.port.custom.origin <= port.range <= spark.service.port.custom.origin + spark.port.maxRetries

e.g. spark.service.port.custom.origin=49152 and spark.port.maxRetries=10, the actual range of the port is greater than or equal to 49152 and less than 49163

Why are the changes needed?

In a real production environment, there are some special scenarios where we need to limit the scope of the service port.

Does this PR introduce any user-facing change?

No

How was this patch tested?

new ut

chong0929 avatar Aug 30 '22 06:08 chong0929

Can one of the admins verify this patch?

AmplabJenkins avatar Aug 30 '22 10:08 AmplabJenkins

@HyukjinKwon Hi,would you mind to take another look?

chong0929 avatar Sep 05 '22 06:09 chong0929

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

github-actions[bot] avatar Dec 26 '22 00:12 github-actions[bot]