A question on hyperparameters

[ocrinz]

Hi,

Thanks for your great work!

I have trouble reproducing your results with GIN on Cora under 20% Metattack. I used the pre-perturbed data provided by DeepRobust with Pro-GNN splits (data and splits). However, I only got 58.22±4.04 (10 reps), which is far from 72.2 in your paper. Even if the dataset split in Pro-GNN is different from yours, I don't think the results should be so different.

So I wonder if I made mistakes about the hyperparameters. I followed the hyperparameters in your paper and your code:

    epochs = 200,
    patience = 10,
    lr = 0.01,
    weight_decay = 5e-4,
    hidden = 16,
    dropout = 0.5,
    modelname = 'GIN',
    GNNGuard = True,
    seed = 15,

Could you take a look and see which values of hyperparameters should I use? Thanks.

P.S. To save memory, I changed cosine_similarity in defense/gin.py:

GNNGuard/defense/gin.py

Lines 157 to 159 in 33f5390

	sim_matrix = cosine_similarity(X=fea_copy, Y=fea_copy) # try cosine similarity
	# sim_matrix = torch.from_numpy(sim_matrix)
	sim = sim_matrix[row, col]

to the following:

from sklearn.preprocessing import normalize

def paired_cosine_similarity(X, i, j):
    X_normed = normalize(X)
    return (X_normed[i] * X_normed[j]).sum(1)

sim = paired_cosine_similarity(fea_copy, row, col)

I think this code should be equivalent to yours.