dgl icon indicating copy to clipboard operation
dgl copied to clipboard

How Can I Prefetch specific nodes' feature in the training of minibatch

Open zhangnuc opened this issue 2 years ago • 4 comments

I read the code tell me the prefetch specified by dataloader can only select all the src nodes of the first layer GNN of the preloaded blocks. I want to load the selected nodes (maybe these nodes do not appear in the minibatch generated by this sampling), how should I achieve this? (This idea is about the paper : PaGraph: Scaling GNN Training on Large Graphs via Computation-aware Caching )

Looking forward to your answer , thanks u veryyyyy muuuuuch !

zhangnuc avatar Sep 10 '22 11:09 zhangnuc

@BarclayII Could you take a look?

jermainewang avatar Sep 13 '22 01:09 jermainewang

So far the prefetching pipeline only supports loading all ndoe features. Maybe you could check out the official code for your specific setting (https://github.com/zhiqi-0/PaGraph) which is also based on DGL?

BarclayII avatar Sep 19 '22 06:09 BarclayII

I got it ! thks a lot ! !! Also the version of this code is about dgl 0.4.1 , which have some function deleated in dgl0.9.1 . Will these function be republished in the future ?

zhangnuc avatar Sep 21 '22 00:09 zhangnuc

We are unlikely to add the deleted functions back to later versions. Which of the functions do they use that don't appear in 0.9.1? Maybe I could give you pointers for replacements.

BarclayII avatar Sep 26 '22 06:09 BarclayII

This issue has been automatically marked as stale due to lack of activity. It will be closed if no further activity occurs. Thank you

github-actions[bot] avatar Oct 27 '22 01:10 github-actions[bot]

Hi zhangnuc, I am closing this issue assuming you are happy about our response. Feel free to follow up and reopen the issue if you have more questions with regard to our response.

frozenbugs avatar Oct 27 '22 05:10 frozenbugs