mac studio can't generate token

[arthasyou]

the gpt.py code well run at cuda. but when I can devices with mps. this model can be trained. but can not generate token. when run the code at generating can't finish. keeping running forever. thx for reply.

[arthasyou]

but when I change mps as cpu generating can work well but too slow

[fasterinnerlooper]

Does it run forever or does it just take a really long time? I ask because the most efficient way of training and running inference on a model like this is to use a CUDA-based GPU. Even the MPS chips in Macs are still not as efficient as a CUDA-based GPU, though they are more efficient than a CPU. The other side of things is that the MPS support in PyTorch is changing everyday, so make sure you're using the most up-to-date version of PyTorch when you're training and running inference on your model. My knowledge of debugging LLMs is lacking so I can't provide any way to find out if the model you produce is having computational issues or if it's just taking a really long time, but this could be an avenue you could go down if you're certain that you're code is working as intended. I'd be interesting in seeing how things go for you as my workplace provides us with MacBooks, and I have yet to try training a model on this specific hardware.