geni

geni copied to clipboard

Adds partial Delta Lake support

3

Adds support for [Delta Lake](https://delta.io/) storage format. This PR is best reviewed 1 commit at a time. Currently, some core functionality of Delta is disabled due to a [known incompatibility...

erp12

- [x] I have read through the quick start and installation sections of the [README](../README.md). ## Info | Info | Value | | --- | --- | | Operating System...

gnarroway

Possible issue with example 2.5 from cookbook

2

- [X] I have read through the quick start and installation sections of the [README](../README.md). ## Info | Info | Value | | --- | --- | | Operating System...

bhougland18

Should Geni have support for Delta Lake or should that be a separate library?

6

[Delta Lake](https://delta.io/) brings a lot of crucial features into the Spark ecosystem. Some of the highlights include: 1. ACID transactions. 2. Time travel between data versions. 3. Safe in-place updates,...

erp12

Support Spark NLP ?

4

There is a pre-release for Spark NLP on Spark 3.0.0 This might it possible to use it with Geni ? https://github.com/JohnSnowLabs/spark-nlp/releases/tag/3.0.0-rc8

behrica

GraphFrames package with Geni?

4

Is it feasible to use a sparkml java package like https://graphframes.github.io/graphframes/docs/_site/index.html with Geni? Or is this outside of the scope?

samrose

ArityException with cookbook sample 4

5

- [x] I have read through the quick start and installation sections of the [README](../README.md). ## Info | Info | Value | | --- | --- | | Operating System...

jriekhof

Creating histograms is a very common activity. Geni offers [`cut`](https://github.com/zero-one-group/geni/blob/develop/src/clojure/zero_one/geni/core/foreign_idioms.clj#L158) which supports the creation of histograms as a function of `bins`, an array of values, but the user has to...

andres-moreno

Geni users would benefit from support for Spark User Defined Functions on dataframes as documented [here](https://spark.apache.org/docs/latest/sql-ref-functions-udf-scalar.html). UDFs are very useful for data analysis from the simple classification of continuous values...

andres-moreno

spark-session cannot be changed (any more ?)

5

Following the minikube guide: https://github.com/zero-one-group/geni/blame/develop/docs/kubernetes_basic.md the verification of line 118 fails. It seems that I cannot change the spark-session, by calling `g/create-spark-session` I am pretty sure, that it worked at...

behrica