Subway icon indicating copy to clipboard operation
Subway copied to clipboard

support graph 64-bit edge number

Open jdongca2003 opened this issue 3 years ago • 0 comments

In the master branch the graph edge number is limited to uint32. In this pull request,

  1. change all nodePointer to ull (unsigned long long (64-bit))
  2. change all node data type to uint (typedef unsigned int uint --- 32-bit uint32). Both are defined in globals.hpp. Currently uint32 is still used to save GPU RAM. But it is easy to support 64-bit node data type. Just refine uint
  3. for converter, use stdin as input. This helps to support very large graph where edge list is compressed (clueweb12).

I successfully tested the version on a subset of clueweb12 graph (http://www.lemurproject.org/clueweb12/webgraph.php/) : 27 billion edges and 600 million nodes on nvidia A100 GPU.

jdongca2003 avatar Oct 26 '22 02:10 jdongca2003