spark-on-kubernetes-helm icon indicating copy to clipboard operation
spark-on-kubernetes-helm copied to clipboard

Published 20 hours ago • verified publisher icon JahstreetOrg

Upgrade to Spark 3.2.0

Open Rustem opened this issue 3 years ago • 1 comments

Hi @jahstreet how are you doing so far ? First of all thank you for your initiatives. This is awesome. What is the current status of the project ? We are considering to leverage your work for our analytics platforms. We are thinking to use spark distribution 3.2.0 and hadoop 3.2. What would be your suggestion in such case?

Rustem avatar Feb 15 '22 00:02 Rustem

Hi Rustem. Thank you for your feedback. I hope to get to the project in the coming month or 2 when would jump on the relevant data platform project as well ;)

What would be your suggestion in such case?

Basically you can use the existing build as a reference and patch it accordingly. There might be some dependency conflicts in Livy package resolving which might be the most complex step.

jahstreet avatar Feb 15 '22 10:02 jahstreet

Closing this issue in favour of https://github.com/JahstreetOrg/spark-on-kubernetes-helm/issues/82 within which it will be resolved, lets continue there.

And thanks for the patience 🙏 .

jahstreet avatar Oct 20 '22 07:10 jahstreet