pythia
pythia copied to clipboard
`torch.concat` is supported when reproducing results with docker
Many thanks for the kind sharing!!
When reproducing training results, the docker file specifies torch==1.8.1. However, torch at 1.8.1 had not introduced torch.concat yet, which will cause errors on this line of code:
https://github.com/EleutherAI/gpt-neox/blob/71df4d5017f9f4919566a11454fe3a507ffdc632/megatron/model/transformer.py#L432
Simply changing torch.concat to torch.cat can make it compatible.