[Google Colab] RuntimeError: CUDA error: an illegal memory access was encountered

[AJroky]

Hey guys, I'm a beginner to this and, I have used the Colab notebook several times to finetune a llama3 model.

For the past 4 days, I have been getting this CUDA Runtimeerror, when I try to use the Colab notebook.

Can anyone help?

[danielhanchen]

@AJroky Most likely out of memory - are you using the notebooks we posted on Github without changing them?

[AJroky]

@danielhanchen yes, it's the same notebook. I had made only 2 changes,

• The dataset is my own
• 500 steps, instead of 60 steps

However, about a week ago I used the same notebook with these changes (mentioned above) and was able to fine-tune. But this week when I tried, I kept getting this CUDA error.

[danielhanchen]

Do you know what the GPU memory usage is? (It's located on the right sidebar and click View Resources)

[AJroky]

@danielhanchen Apologies for not mentioning it earlier, the GPU was the Nvidia Tesla T4 (I was using the free tier of Colab)

[danielhanchen]

Ok thats weird - tbh im stuck since I don't have error logs - I'm just guessing it went out of memory

[AJroky]

Ok understood. Let me try to using a different GPU then. Thanks for the assist. 👍

[danielhanchen]

@AJroky Wait can you retake your dataset and do it in the Colabs we have on our Github page (ie retake our Colabs without editting them, the edit them again)