Alexandre Archambault

Results 365 comments of Alexandre Archambault

:+1: A new release would also include https://github.com/aws/aws-sdk-java/pull/590, which would be cool.

Yes, some APIs changed in Ammonite recently. It should work with Ammonite `1.5.0` or `1.6.0` (but not Ammonite >= `1.6.1`). I hope to propose some changes in Ammonite very soon,...

As you said, ammonite-spark currently relies on `replApi`, which is only accessible from the repl, not from scripts. The things it accesses through `replApi`, mostly `replApi.sess.frames` could be made accessible...

FYI, [this gist](https://gist.github.com/alexarchambault/8e1eec124f00b1c5576a29899ae39569) lists commands to get ammonite-spark up-and-running with an EMR cluster. I'd like to make it an actual tutorial, but didn't find the time to do that yet.

cc @mpacer who was also interested by that (link in my previous comment)

@mycaule Did you add the extra dependencies _before_ creating the Spark session? (or [call `AmmoniteSparkSession.sync()`](https://github.com/alexarchambault/ammonite-spark#syncing-dependencies) else)

I think that's all there is to it… It's mentioned [here](https://github.com/alexarchambault/ammonite-spark#getting-orgapachesparksqlanalysisexception-when-calling-tods) too. I'd really like to automate that.

@rdblue Neat. Would be it nice if spark allowed to add that, yes.

Yep, I confirm it behaves this way currently, and I agree this ought to be fixed… (There's the same problem in Jupyter classic, when one clicks on the stop button.)...