Make reindexing compatible to Spark RDDs
Currently, one can use
new InputFormatRDD().readGraphRDD(new MapConfiguration(configurationMap), sparkContext)
to load the entire graph to Spark as Spark RDDs. FYI I also have a PR for TINKERPOP-2772 to make Spark load more accessible to users.
One can do any custom analysis on these RDDs, but it becomes difficult when one wants to run reindexing in Spark. The curse is, these RDDs are in TinkerPop's StarGraph.StarVertex type and thus not acceptable by IndexSerializer::reindexElement method. I am not sure if this is doable but it would be awesome if we could refactor reindexing APIs to make them accept any kind of Vertex rather than just JanusGraphVertex.
FWIW, the workaround I use in my project is to only use the vertex id in the RDD and query JanusGraph to retrieve the vertex, which is not that efficient and is definitely a waste of resources.