Questions on concatenation of vectors in the code

[skx300]

In the paper, the current node's vector is concatenated with the aggregated neighbors' vector, then it is fed into a fully connected (dense) layer. However, reversely, in the code (aggregators.py), the current node's vector and aggregated neighbors' vector are fed into their own dense layers respectively, then results of them are concatenated. Could you tell me why? Thank you in advance.

[preetham-salehundam]

@skx300 I have the same question as well, Did you figure out by any chance?

[skx300]

@preetham-salehundam I haven't figured out that. Have you got any idea? I think it is necessary to feed the concatenated vector into one dense layer, no matter whether they have got into their own one or not before.

[neilaronson]

I'm not sure, but this could simply be so that if you are using mean as the aggregator but also want to use dropout, you have a chance to do so. Otherwise the node representations don't pass through a final dropout before predictions.

[skx300]

I think I may get the point. The dense layer (which is a matrix weight W^k), multiplied by the concatenated vector of node's itself and its neighbors, can be split into two dense layers (two weight matrices, we could let the first half of W^k as W^k1 and the second half as W^k2), then the node's itself can be multiplied by W^k1, and its neighbors can be multiplied by W^k2. The concatenation of the result of them are identical to the aforementioned one (which concat first and then multiply ). Since the nonlinear activation function is applied at the end, it doesn't matter, they are just linear operations.