Kevin Kuo

AWS scripts to launch EMR clusters for use with sparklyr

2

Feature request from webinar. We currently have a tutorial to go through the console at https://spark.rstudio.com/articles/deployment-amazon.html.

Something to investigate after the current `ml` functionality as been ported. We're currently sending these to R numeric vectors, which might not be a good idea for large sparse vectors...

https://issues.apache.org/jira/browse/SPARK-7132

https://issues.apache.org/jira/browse/SPARK-22119

https://issues.apache.org/jira/browse/SPARK-10697

https://issues.apache.org/jira/browse/SPARK-14682

https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameNaFunctions

einops

2

https://github.com/arogozhnikov/einops

Currently in the R package there's no way to apply a module along a dimension except for when the module is `nn_linear`. possible impl: https://pytorch.org/docs/master/generated/torch.vmap.html https://github.com/zou3519/functorch