A question on hyperparameters

[ocrinz]

Hi,

Thanks for your great work!

I have trouble reproducing your results with GIN on Cora under 20% Metattack. I used the pre-perturbed data provided by DeepRobust with Pro-GNN splits (data and splits). However, I only got 58.22±4.04 (10 reps), which is far from 72.2 in your paper. Even if the dataset split in Pro-GNN is different from yours, I don't think the results should be so different.

So I wonder if I made mistakes about the hyperparameters. I followed the hyperparameters in your paper and your code:

    epochs = 200,
    patience = 10,
    lr = 0.01,
    weight_decay = 5e-4,
    hidden = 16,
    dropout = 0.5,
    modelname = 'GIN',
    GNNGuard = True,
    seed = 15,

Could you take a look and see which values of hyperparameters should I use? Thanks.

P.S. To save memory, I changed cosine_similarity in defense/gin.py:

https://github.com/mims-harvard/GNNGuard/blob/33f5390bb5adff3f8bed5bcb01e9ba50e2c74c38/defense/gin.py#L157-L159

to the following:

from sklearn.preprocessing import normalize

def paired_cosine_similarity(X, i, j):
    X_normed = normalize(X)
    return (X_normed[i] * X_normed[j]).sum(1)

sim = paired_cosine_similarity(fea_copy, row, col)

I think this code should be equivalent to yours.