[Docs] [Core] [Data] Spark integration via RayDP

[dynamicwebpaige]

trafficstars

Description

Motivation

It can be challenging for users to understand how to use Spark in combination with Ray, via something like RayDP; and the documentation on both our website and in the raydp website is minimal.

Feature Request

We should minimally have:

• An architecture diagram discussing how to use Spark on Ray.
• A tutorial showing how to migrate Spark + MLlib workloads to use Ray.
• A discussion of the benefits and trade-offs for preferring Spark-on-Ray rather than Spark.

This table might be helpful, though is from 2020 and could perhaps need to be updated:

Use case

No response

[stale[bot]]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[stale[bot]]

Hi again! The issue will be closed because there has been no more activity in the 14 days since the last message.

Please feel free to reopen or open a new issue if you'd still like it to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for opening the issue!