sparklyr icon indicating copy to clipboard operation
sparklyr copied to clipboard

sparklyr doesn't work with sparkxgb

Open ciasom opened this issue 5 years ago • 1 comments

After loading sparkxgb (0.1.0), sparklyr (v1.0.1) can't connect to spark (via livy). It failed in the following piece of code in function spark_connect:

if (length(scon$extensions) > 0) { for (initializer in scon$extensions$initializers) { if (is.function(initializer)) initializer(scon) } }

where Browse[4]> scon$extensions [1] "sparkxgb" Browse[4]> class(scon$extensions) [1] "character"

So scon$extensions$initializers throws exception of "$ operator is invalid for atomic vectors".

ciasom avatar Jun 06 '19 04:06 ciasom

The same issue occurs when loading the "sparklyr.nested" package prior to connecting to spark.

tschutte avatar Sep 12 '19 15:09 tschutte