Andrew Li

[BUG] T5 extended attention mask shape mismatch with transformer engine

5

**Describe the bug** Running the most recent version of the T5 pretraining script out of the box raises a Value Error, particularly in the following line: ``` [rank0]: File "/home/miniconda3/lib/python3.12/site-packages/transformer_engine/pytorch/attention.py",...