cugraph
cugraph copied to clipboard
[FEA] Buffered and In-Memory Sampling
Adds support for buffered sampling, and supports in-memory sampling as a form of buffered sampling. Also separates the sample readers and sample writers into their own files to make the code easier to read.
Adds support for buffered in-memory sampling in cuGraph-PyG. Updates the PyG examples to add in-memory sampling as an option.
Also fixes a bug where COO output of the distributed sampler (which is rarely used) was overwriting previous batches.
Based on benchmarks on H100, buffered in-memory sampling is 3.2x faster than disk sampling. This results in an overall speedup of 1.25x. This speedup is consistent on SG, MG, and MNMG up to 16 GPUs for ogb-papers100M.
Closes rapidsai/cugraph-gnn#14