GraphScope
GraphScope copied to clipboard
Implement local vertex map
In order to mitigate the pressure of dealing with graphs with billions of vertices.
Since the global vertex map implementation saves a copy of all vertices in each partition, scale up won't help.
solution
- shuffle the edge table first and fetch the outer vertices gid from other worker
- construct the local vertex map base on the local vertices oid and outer vertices gid
sub-task
- [x] implement the arrow local vertex map
- [x] test the runnable and correctness on p2p
- [x] test on graph-view 1B large graph (string oid)