ammonite-spark
ammonite-spark copied to clipboard
alexarchambault
Run spark calculations from Ammonite
## About this PR 📦 Updates [com.lihaoyi:mill-main](https://github.com/com-lihaoyi/mill) from `0.10.12` to `0.11.5` 📜 [GitHub Release Notes](https://github.com/com-lihaoyi/mill/releases/tag/0.11.5) - [Release Notes](https://github.com/com-lihaoyi/mill#changelog) - [Version Diff](https://github.com/com-lihaoyi/mill/compare/0.10.12...0.11.5) ## Usage ✅ **Please merge!** I'll automatically update this...
## About this PR 📦 Updates [org.scala-lang:scala-library](https://github.com/scala/scala) from `2.13.11` to `2.13.12` 📜 [GitHub Release Notes](https://github.com/scala/scala/releases/tag/v2.13.12) - [Version Diff](https://github.com/scala/scala/compare/v2.13.11...v2.13.12) ## Usage ✅ **Please merge!** I'll automatically update this PR to resolve...
## About this PR 📦 Updates [org.eclipse.jetty:jetty-server](https://github.com/eclipse/jetty.project) from `9.4.51.v20230217` to `12.0.0` ## Usage ✅ **Please merge!** I'll automatically update this PR to resolve conflicts as long as you don't change...
## About this PR 📦 Updates [org.scala-lang:scala-library](https://github.com/scala/scala) from `2.12.11` to `2.12.18` 📜 [GitHub Release Notes](https://github.com/scala/scala/releases/tag/v2.12.18) - [Version Diff](https://github.com/scala/scala/compare/v2.12.11...v2.12.18) ## Usage ✅ **Please merge!** I'll automatically update this PR to resolve...
Seems the Spark REPL support for Spark Connect relies on Ammonite (https://github.com/apache/spark/pull/40515, https://github.com/apache/spark/pull/40675, https://github.com/apache/spark/pull/40894). [Their initialization code](https://github.com/apache/spark/blob/1c713dbacc5074e7812ba48b4cebf51309e2fb84/connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/application/ConnectRepl.scala#L81-L89) looks pretty straightforward.
How do we read from S3 using Spark Standalone mode? Has anybody been able to get this to work? I have been able to get the dependencies work when using...
with same env, I can start spark session with yarn mode in spark-shell, but failed with ammonite-spark due to permission issue permission error log ``` org.apache.hadoop.security.AccessControlException: Permission denied: user=ai_user, access=WRITE,...
## About this PR 📦 Updates [org.eclipse.jetty:jetty-server](https://github.com/jetty/jetty.project) from `9.4.54.v20240208` to `12.0.9` ## Usage ✅ **Please merge!** I'll automatically update this PR to resolve conflicts as long as you don't change...
## About this PR 📦 Updates [org.scala-lang:scala-library](https://github.com/scala/scala) from `2.13.11` to `2.13.14` 📜 [GitHub Release Notes](https://github.com/scala/scala/releases/tag/v2.13.14) - [Version Diff](https://github.com/scala/scala/compare/v2.13.11...v2.13.14) ## Usage ✅ **Please merge!** I'll automatically update this PR to resolve...
## About this PR 📦 Updates [com.lihaoyi:utest](https://github.com/com-lihaoyi/utest) from `0.8.2` to `0.8.3` 📜 [GitHub Release Notes](https://github.com/com-lihaoyi/utest/releases/tag/0.8.3) - [Version Diff](https://github.com/com-lihaoyi/utest/compare/0.8.2...0.8.3) ## Usage ✅ **Please merge!** I'll automatically update this PR to resolve...