Add `Polynormer` model and example [1/n]

[xnuohz]

Issue

• From #9751

Usage

python examples/polynormer.py --post_bn

Highlight

First round of benchmarks run on NVIDIA internal computers

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 85.71%. Comparing base (c211214) to head (ed3f1d5). :warning: Report is 117 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9908      +/-   ##
==========================================
- Coverage   86.11%   85.71%   -0.40%     
==========================================
  Files         496      501       +5     
  Lines       33655    34454     +799     
==========================================
+ Hits        28981    29532     +551     
- Misses       4674     4922     +248     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[xnuohz]

batch = 1

file	model	valid acc	test acc	avg training time	avg inference time
ogbn_train	Polynormer	72.81%	71.18%	29.1350	4.3969
ogbn_train_cugraph	Polynormer	72.76%	70.95%	32.5175	23.7977
ogbn_train_cugraph_multigpu	Polynormer	71.35%	70.24%	32.2606	23.8230

ogbn_train_cugraph_multi_gpu_polynormer.log ogbn_train_cugraph_polynormer.log ogbn_train_polynormer.log

[xnuohz]

1. Polynormer also supports batch
2. update ogbn_train.py and follow the paper's training style
3. add an error message for the cugraph example before cugraph support disjoint
4. my resource is limited, so batching graph transformer is hard to train and get a result

[puririshi98]

@Kh4L any reviews welcome as well

[xnuohz]

the 3-layer polynormer is not as good as gcn, while the original paper uses 7 layers ogbn_polynormer.log

[puririshi98]

the 3-layer polynormer is not as good as gcn, while the original paper uses 7 layers ogbn_polynormer.log

@xnuohz since this example framewokr is not just for polynormer lets leave default at 3, and add an if model==polynormer and num_layers!=7: print("The original polynormer paper recommends 7 layers, you have chosen", num_layers, " which may effect results. See for details")

[xnuohz]

@puririshi98 @Kh4L ready for review, thanks:)

[puririshi98]

need to test and then ill ask matthias for review to avoid similar mistakes to the last PR

[xnuohz]

ogbn_polynormer.log

[xnuohz]

@puririshi98 ready for review

[xnuohz]

@rusty1s thanks for your review, resolved and increased code coverage. linting error not related to my changes.

[puririshi98]

@xnuohz can you see if you can fix the CI issue it might be that the explainer test runs somethjing based on models available in pyg and you have added a new model

[xnuohz]

@puririshi98 waiting #10310 to fix and merge also checkout this pr, same ci error as mine

[puririshi98]

@puririshi98 waiting #10310 to fix and merge also checkout this pr, same ci error as mine

okay perfect analysis, this isnt on you. i will merge this once those go in and CI becomes green

[xnuohz]

hi @puririshi98 CI becomes green, can you merge?

[xnuohz]

CI may fail when torch.allclose uses default parameters, changing these default parameters allows CI to pass