Javier Arturo Porras Luraschi

@statmatt by bad, I think `g` is case sensitive (yet another thing to consider streamlining in `sparklyr` config) and parameter should be `sparklyr.shell.driver-memory`. ```r sc

@schlichtanders how would these libraries be used? Could you give us an example? Is this perhaps related to `spark_apply()`? Even if the libraries were included, it is not clear to...

@fogongzi it's possible that the packages are being downloaded and is taking too long... try changing the initialization timeout: ```r config

This is reasonable, marking as feature request. Currently we only set the warehouse directory for windows OS, see https://github.com/rstudio/sparklyr/blob/ff105bcf10378c683c116480256404316f9aed2a/R/install_spark.R#L196. Reason being that the default path is `c:\tmp\hive` which causes install...

Repro: ```r library(sparklyr) library(DBI) library(dplyr) sc

@gillesgerlo it is supported; however, it is more efficient to use `dplyr` without `spark_apply()`, we recommend using `sparklyr` as follows: ```r iris_tbl %>% group_by(Species) %>% summarise(uniqueValues=n_distinct(Sepal_Length)) ``` If you really...

Ah, this would be really interesting! So, Spark recently introduced the concept of a **barrier** which is meant to be used with deep learning workflows but is generic enough that...

CC: @edgararuiz

Right, it would be great to be able to run something like: ```r sdf_len(sc, 10) %>% invoke("map", ensure_scala_function("(n:Double) => n + 1")) ``` Seems like this shouldn't be that hard...

https://github.com/rstudio/shinytest makes this feature request more compelling, I think. CC: @wch