Da Zheng comments

Results 38 comments of


                                            Da Zheng

Create documentation on estimate hardware requirement for training GNN models on large graphs.

https://gist.github.com/zheng-da/ca33af9048d09553f668f95bfd13cb0e the code shows how to calculate the memory requirement for GNN training in DistDGL.

Distributed partitioning pipeline should be flexible to generate specified the number of partitions

Decouple the number of processes and the number of partitions. The number of processes is 10, the number of partitions is 10 * k.

Reduce the peak memory in DistDGL

i have itemized the work. i think we only need to fix the 4 items i listed above.

[Call For Discussion] Shall we set the default value of remove_target_edge_type to False?

if the edge classification is to predict the existence of an edge between two nodes, we should remove the target edge in the message passing to avoid information leak. if...

Balance the training/validation/test set in graph partitioning

https://github.com/awslabs/graphstorm/pull/714

Multi-processing error when dataset is large and using distributed node embeddings

can you try `num_samplers=0`?

Multi-processing error when dataset is large and using distributed node embeddings

is it an out-of-memory error?

Rename training_scripts and inference_scripts folders

is this still needed?