graphtrans

graphtrans copied to clipboard

Question about padding for transformer

1

Hi, Thanks for your excellent work. I found the implementation of padding a batch in your code is confusing. The code for batch is: ```python def pad_batch(h_node, batch, max_input_len, get_mask=False):...

agave233

Hi! I'm trying to scale the GNN transformer model down so I can run it with less compute. I can change the GNN embedding dimension, number of layers, transformer feedforward...

inwaves

Hello, Thanks for excellent work. But I have found some possible problems. In the paper, the authors mention that "In Table 5, we tested several common methods to for sequence...

LUOyk1999

I think it should be a^{l}_{v,w} = softmax(a^{l}_{v,u} )

Irish-kw

masked_transformer_encoder.py, line47, should be changed to > att = att.masked_fill(valid_input_mask.unsqueeze(1).unsqueeze(2) != 0, mask_value) the origin code means that change all the place where ```valid_input_mask==False``` into mask_value

JizeZhangCS

## Trigger method: - Run python main.py --configs ./configs/NCI1/gnn-transformer/no-virtual/gd\=128+gdp\=0.1+tdp\=0.1+l\=3+cosine.yml (i.e. NCI1, small GCN) - change the num_encoder_layers: 3 into 0 in this file - add a new line: num_encoder_layers_masked: 3...

JizeZhangCS