spark icon indicating copy to clipboard operation
spark copied to clipboard
verified publisher icon dotnet

support Apache Spark 3.4

Open Ponyo1234 opened this issue 2 years ago • 4 comments

Is your feature request related to a problem? Please describe. In Apache Spark 3.4, Spark Connect introduced a decoupled client-server architecture that allows remote connectivity to Spark clusters using the DataFrame API and unresolved logical plans as the protocol

Describe the solution you'd like support spark-3.4

Ponyo1234 avatar Aug 24 '23 01:08 Ponyo1234

[30thAug2023] .NET for Apache Spark documentation will no longer be updated

GeorgeS2019 avatar Aug 30 '23 22:08 GeorgeS2019

I’ve been looking into this separately and believe it is possible and probably the right thing to do going forward, we are waiting to find out about the status of the project but I am going to start working on this.

GoEddie avatar Sep 21 '23 12:09 GoEddie

@GoEddie

Do follow @luisquintanilla contribution history with Spark.NET before it was even called what we have today. He went all the way back with the project.

Once he is showing interest.

This will be the new beginning!

Thank you for your interest. My role will be more of a morale boosting rather than hands-on.

GeorgeS2019 avatar Sep 21 '23 12:09 GeorgeS2019

Thanks @GeorgeS2019 Ive let Luis know on the other issue I’m interested in helping with the project!

GoEddie avatar Sep 21 '23 12:09 GoEddie