pyg_autoscale icon indicating copy to clipboard operation
pyg_autoscale copied to clipboard

Published 20 hours ago • verified publisher icon rusty1s

Results on OGBN-Papers100M

Open RX28666 opened this issue 10 months ago • 6 comments

Hello,

I am wondering if there are any results of GAS on OGBN-Papers100M. (Or results on some datasets larger than ogbn-products)

BTW, since the dataset is so big, preprocessing steps such as partitioning with METIS are unrealistic to implement as usual. I am also wondering if there are any code scripts I can refer to that can help solve this issue.

Thanks.

RX28666 avatar Apr 06 '24 16:04 RX28666

We didn't test on ogbn-papers100M at this point in time. Given enough RAM, this should definitely work though. I partially agree that METIS becomes infeasible on larger datasets (although all large-scale graph learning variants rely on it to scale), but you can also fallback to random partitioning.

rusty1s avatar Apr 07 '24 07:04 rusty1s

Thanks for your advice, I will try to run GAS on ogbn-papers100m.

RX28666 avatar Apr 07 '24 16:04 RX28666

Hello Matthias,

I also faced a issue when I try to install the package, recently I updated my CUDA to 12.1:

CUDA Version: 12.1
PyTorch Version: 2.2.1+cu121

I tried to install the package in both ways provided, they all returned:

RuntimeError:
      The detected CUDA version (11.6) mismatches the version that was used to compile
      PyTorch (12.1). Please make sure to use the same CUDA versions.

Is this because the current package doesn't support cuda 12.1? Thanks.

RX28666 avatar Apr 07 '24 17:04 RX28666

You need to re-install this package if you also update your CUDA version.

rusty1s avatar Apr 08 '24 11:04 rusty1s

Hello,

I deleted the original one by:

pip uninstall pyg_autoscale
WARNING: Skipping pyg_autoscale as it is not installed.

then re-install using pip install git+https://github.com/rusty1s/pyg_autoscale.git and also tried python setup.py install

they both returned the same bug.

The detected CUDA version (11.6) mismatches the version that was used to compile
      PyTorch (12.1). Please make sure to use the same CUDA versions.

Is there anything I was missing? Any help would be appreciated.

RX28666 avatar Apr 08 '24 14:04 RX28666

Ok, got it. This is IMO expected. You are using your local CUDA version (11.6) to compile this package, while you have PyTorch installed with CUDA version 12.1. What you can do

  • Downgrade PyTorch to use CUDA 11.*
  • Alternatively, update your local CUDA version to 12.1

rusty1s avatar Apr 11 '24 10:04 rusty1s