Carson Wang

LGTM

You only need to init once. If you see this error, it is because the pervious driver is not closed properly. You can find the Java process ID and kill...

@Bowen0729 , can you please share more about your use case and requirement?

This sounds like an issue. So two users will run their Spark driver on the same node and connect to the same Ray cluster, right? Are you interested in submitting...

Great! Thanks!

@yanivg10 , did you check the Ray dashboard? How many cores did Ray detect after running `ray.init()`? RayDP will create actors based on the parameters you gave to `raydp.init_spark`. It...

@kira-lin , it is not quite friendly to force users specifying the code_search_path when using RayDP. Instead of setting code_search_path when init Ray, is it possible to create the RayAppMaster...

You can try to tune a few Spark configurations to see if you can run it successfully with less memory. For example, you can increase "spark.sql.shuffle.partitions". You can also allocate...

@Hoeze , Thanks for the feedback! I am curious what your entire pipeline looks like and what operations are needed after the Spark processing? We also support creating a Ray...

Thanks for the detailed explanation. This makes sense. We should probably add an example for this.