Subway
Subway copied to clipboard
support graph 64-bit edge number
In the master branch the graph edge number is limited to uint32. In this pull request,
- change all nodePointer to ull (unsigned long long (64-bit))
- change all node data type to uint (typedef unsigned int uint --- 32-bit uint32). Both are defined in globals.hpp. Currently uint32 is still used to save GPU RAM. But it is easy to support 64-bit node data type. Just refine uint
- for converter, use stdin as input. This helps to support very large graph where edge list is compressed (clueweb12).
I successfully tested the version on a subset of clueweb12 graph (http://www.lemurproject.org/clueweb12/webgraph.php/) : 27 billion edges and 600 million nodes on nvidia A100 GPU.