PowerGraph icon indicating copy to clipboard operation
PowerGraph copied to clipboard

issue on parallel ingress using obvious heuristics

Open weristdas opened this issue 11 years ago • 4 comments

It seems that thers is no general thread-level parallel support for this:

file: ./graphlab-master/src/graphlab/graph/distributed_graph.hpp 1902 #ifdef _OPENMP 1903 #pragma omp parallel for 1904 #endif 1905 for(size_t i = 0; i < graph_files.size(); ++i) { 1906 if ((parallel_ingress && (i % rpc.numprocs() == rpc.procid()))

When loading from multiple files by multiple processes on 'obvious mode', openmp doesn't work correctly here.

weristdas avatar Aug 12 '13 05:08 weristdas

Hmm... Obvious mode?

ylow avatar Aug 13 '13 17:08 ylow

yes, e.g. run pr like this: mpirun -n 16 --hostfile ~/slots ~/bin/pagerank --graph_opts="ingress=oblivious" --graph /mnt/nfs/twitter/ --format tsv

在2013-08-14 01:46:53,闫洁[email protected]写道:

Hmm... Obvious mode?

— Reply to this email directly or view it on GitHub.

weristdas avatar Aug 14 '13 00:08 weristdas

We need a bit more details on this issue. How many files are in the directory /mnt/nfs/twitter ? And in what way does it fail?

ylow avatar Aug 15 '13 21:08 ylow

Can you provide some output logs?

ylow avatar Aug 15 '13 21:08 ylow