DIG icon indicating copy to clipboard operation
DIG copied to clipboard

The results do not match the description in Readme.md

Open huayangcs opened this issue 1 year ago • 0 comments

hello, thanks for this amazing benchmark work. However, when I reproduced Pgexplainer's results using the data you provided, the trained model you provided, and the pgexplainer model you provided, I found that the results were significantly different from the ones you provided in Readme.md. May I ask the possible reasons for this result? Thanks for your help.

DATASETS=ba_2motifs
python -m benchmarks.xgraph.pgexplainer_edges datasets=$DATASETS explainers=pgexplainer

results are :

Fidelity: 0.4920 Fidelity_inv: 0.4253 Sparsity: 0.5066 Accuracy: 0.04254115393283751 Stability: nan

after I change the config.explainers.sparsity = 0.8, the results are

Fidelity: 0.3911 Fidelity_inv: 0.4889 Sparsity: 0.8089 Accuracy: 0.04254115393283751 Stability: nan

huayangcs avatar Apr 07 '23 12:04 huayangcs