Why is the node_feature matrix + 1 larger than the largest index of nodes?

[benelot]

In the preprocess code you currently only use a zero matrix, so it does not matter in your experiments. https://github.com/twitter-research/tgn/blob/2aa295a1f182137a6ad56328b43cb3d8223cae77/utils/preprocess_data.py#L74 But why is the feature matrix one row larger than the highest index of the nodes? To me that looks like it is one row larger than the number of nodes.

max_idx = max(new_df.u.max(), new_df.i.max())
rand_feat = np.zeros((max_idx + 1, 172))

In our own dataset, we have actual node features and we try to estimate the benefit of using a TGN to classify the nodes vs. a neural network using only the node features. So I am wondering if I have to include a row of features that are all zero above or below my features to complete the matrix correctly?

[josephKhoury95]

Hello @emalgorithm @benelot, I also have node features were you able to figure out the reason for the +1? Also, did you face any memory problem when adding new features? like "something wrong in how the memory was updated"?

See below, where it is failing at: https://github.com/twitter-research/tgn/blob/2aa295a1f182137a6ad56328b43cb3d8223cae77/model/tgn.py#L166

@emalgorithm, any advise on this? I appreciate any help/reply on this. Thank you.

[benelot]

I could figure it out to be honest. I just added a zero feature to it before and after, 0 mean 1std my features etc but I did not find out any configuration in which the node features even made any difference to the training result. I thought in the end that they might be some preinitialization for the node vector which then are finetuned, but I could neither find anything in the paper supporting that nor any configuration where this really happened.

As far as I remember, I also faced many "wrong memory updates", but not necessarily related to node features. What do you mean by "adding features"? As far as I can remember they are static to each node. So you possibly mean when you replace the zero matrix with your features?

Let me know if you find something or even another temporal graph model combining temporal node & graph features, I am still very interested.

[emalgorithm]

Hi @benelot, the reason for the +1 is that the node indexes start from 0, so if the max index is max_idx, it means that in total we have max_idx + 1 nodes.

I did also observe on some datasets that node features did not make a difference to the performance. I think the model can in principle make use of useful good features, but then it is an empirical question whether the features are actually useful for the task at hand.

[benelot]

@emalgorithm Thanks for the input. Can you give me a hint on how the node features are really used in the model? To me it looks like they serve as some sort of initialization for the node embedding, is that correct? In my experiments, I realized that they were usually just alterated by the node embedding through the graph such that the initialization did not matter. Maybe I overlooked something.

[emalgorithm]

The node feature are actually added to the memory every time the embedding is computed, see here

[benelot]

Oh right, I missed that line! Thx!

[YanivDorGalron]

Hi @benelot, the reason for the +1 is that the node indexes start from 0, so if the max index is max_idx, it means that in total we have max_idx + 1 nodes.

I did also observe on some datasets that node features did not make a difference to the performance. I think the model can in principle make use of useful good features, but then it is an empirical question whether the features are actually useful for the task at hand.

Hey @emalgorithm, thanks for answering questions here :) one thing i didn't understand. you already added +1 to the index [here] so the index actually does not start from 0 at that point