AttributeError: 'TransformerModel' object has no attribute 'bn'

[xinyu-dev]

Thanks for the great study! I was able to run most of the notebooks except that: In Tutorial_Attention_GRN.ipynb, section 2.1:

model.eval()
with torch.no_grad(), torch.cuda.amp.autocast(enabled=True):
    M = all_gene_ids.size(1)
    N = all_gene_ids.size(0)
    device = next(model.parameters()).device
    for i in tqdm(range(0, N, batch_size)):
        batch_size = all_gene_ids[i : i + batch_size].size(0)
        outputs = np.zeros((batch_size, M, M), dtype=np.float32)
        # Replicate the operations in model forward pass
        src_embs = model.encoder(torch.tensor(all_gene_ids[i : i + batch_size], dtype=torch.long).to(device))
        val_embs = model.value_encoder(torch.tensor(all_values[i : i + batch_size], dtype=torch.float).to(device))
        total_embs = src_embs + val_embs
        total_embs = model.bn(total_embs.permute(0, 2, 1)).permute(0, 2, 1)

The line model.bn gave an error 'TransformerModel' object has no attribute 'bn'. Is there a workaround for this? Thanks!