RuntimeError: Expected x1.dtype() == cos.dtype() to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

Open JerryDaHeLian opened this issue 2 years ago • 2 comments

When I Pre-train LLaMA, there is a error: File "/usr/local/lib/python3.8/dist-packages/torch/autograd/function.py", line 551, in apply return super().apply(*args, **kwargs) # type: ignore[misc] File "/home/xxx/TinyLlama/lit_gpt/fused_rotary_embedding.py", line 39, in forward rotary_emb.apply_rotary( RuntimeError: Expected x1.dtype() == cos.dtype() to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

Who can help me? 3q!

Nov 17 '23 06:11 JerryDaHeLian

I have solved it! Look at follows: The bug stem form https://github.com/Lightning-AI/lit-llama/blob/da71adea0970d6d950fb966d365cfb428aef8298/lit_llama/model.py#L130 I managed to change it : from transformers.utils import is_torch_bf16_gpu_available dtype=torch.bfloat16 if is_torch_bf16_gpu_available() else torch.float16,

Jan 13 '24 13:01 haiduo