Are token features and node/type identifiers added or concatenated?

[hmfu]

Thanks for providing the great implementation codes!

After looking into the codes, I have a quick question about the formation of input features to the TokenGT model. Specifically, if I understood the paper correctly, the node features, token identifiers, and token type identifiers are concatenated (C + 2 * d_p + d_e dimensions according to Section 2 - Main Transformer in the paper), while in the code here, they seem to be added together rather than concatenated. Am I misunderstanding the paper or the codes? Or are these two approaches actually equivalent or achieving similar performances?

Thank you for any help on this!

[ryoji-kubo]

Hi, we were also wondering about the same issue and would appreciate it if the authors could provide insight into how the proof will still hold if addition is used instead of concatenation. Thanks!

[xribene]

I'm also interested in any insights. Thanks!

[Jostarndt]

I am also interested in any further information. Thanks! :)