Two users cannot run their Spark driver on the same node and connect to the same Ray cluster.

[Bowen0729]

https://github.com/oap-project/raydp/issues/175#issuecomment-1195190398

two users cannot run their Spark driver on the same node and connect to the same Ray cluster.

Start the java gateway with a random port.

[carsonwang]

Thank you @Bowen0729 , can you please add a test for this?

[Bowen0729]

Thank you @Bowen0729 , can you please add a test for this?

Hi, I have added a test for this.

[Bowen0729]

The CI is failed. What's the problem?

[carsonwang]

The CI is flaky these days. I just triggered a rerun of the failed test.

[Bowen0729]

@carsonwang My test is passed, other tests on macos error.

[carsonwang]

@Bowen0729 Previously I saw you tried to test running two spark drivers on the same node, that is the test we want to have. Is there any challenge to do that?

[Bowen0729]

@Bowen0729 Previously I saw you tried to test running two spark drivers on the same node, that is the test we want to have. Is there any challenge to do that?

Yes, the test can pass on my mac, but cannot pass for the CI, I think I need some help.

[Bowen0729]

Sorry, I've been a little busy recently, so I don't have time to solve this problem. Thanks for helping.

[kira-lin]

@carsonwang the test still hangs sometime. Shall we skip the tests?

[carsonwang]

Ok. Let's skip it for now. Does it also hangs sometime on your local machine?

[kira-lin]

yes, it hangs sometime and when I ctrl-c it, it shows popen_fork, even if I should have set start_method to spawn.

[carsonwang]

Thanks all! Merged