sparkling

sparkling copied to clipboard

Use with AWS Glue

1

Is it recommended to use sparkling with AWS Glue? AWS Glue is a serverless stack on top of Spark provided by EMR. It has some additional features to Spark, but...

WarFox

Adding Spark Streaming support

4

Hi @chrisbetz Thanks for Sparkling, it's delicious. I just started working with it, and my employer Iris.tv is prepared to use it in production. I am going to put some...

blak3mill3r

added spark-session, converts context to spark session

2

I was reading _Spark: The Definitive Guide_ book and running code snippets with **sparkling**, the book starts code examples with **SparkSession** object so I thought it would be nice to...

ertugrulcetin

Yay, it's cool. Now how do I use it with YARN?

3

So how do I use this with YARN? The best I can get out of it is "SparkException: Unable to load YARN support".

robbieh

Reflection warning when using destructuring sugar

2

## The problem I'm observing some Reflection Warnings at the REPL when calling the `s-de/fn` macro, e.g ```clojure (do (set! *warn-on-reflection* true) (s-de/fn [(x ?y)] (+ x (or y 0))))...

vvvvalvalval

Reloaded workflow

2

I have some ideas for setting up sparkling to work in a "reloaded" style but I don't have time to track down the issues right now. Maybe someone has the...

kovasb

Supported features matrix

2

Hello, As a potential user I would really appreciate some kind of supported feature matrix. Something like: | Feature | Support | Notes | | --- | --- | ---...

BorePlusPlus

Add support for Data Frames

25

See https://databricks.com/blog/2015/02/17/introducing-dataframes-in-spark-for-large-scale-data-science.html

chrisbetz

Local namespace symbols referenced from foreach-partition function not found.

5

It would appear that referencing symbols within the current namespace from the function passed to foreach-partition. Oddly enough referencing protocols works, so I originally thought it was specific to helper...

paul-amperity

Connecting a REPL - and why it's not a problem to not being able to run arbitrary code

8

- How to configure nrepl-Server - Security - Serialization explained (esp. mention bindings/closures) - How to work with a Clustered-Apps nrepl and not run into serialization issues.

chrisbetz