Ryan Beppel comments

Repositories
Issues
Comments

Results 4 comments of


                                            Ryan Beppel

Can't fine-tune a model on my dataset in Google Colab

also hitting a similar issue using the docker image.

Question: Are there any work arounds for using DeepSpeed for multi-gpu training

> @rgxb2807 i was hoping accelerate would just work lol It does! and I'm grateful for it. > @rgxb2807 maybe you can raise an issue at the accelerate repository I'll...

Question: Are there any work arounds for using DeepSpeed for multi-gpu training

Raised the issue with the accelerate team here [#2106](https://github.com/huggingface/accelerate/issues/2106)

what(): CUDA error: an illegal memory access was encountered

Also experiencing this regardless of zero stage. Has anyone found a work around? Edit: moving from deepspeed 0.9.3 -> 0.9.5 seems to have resolved my issue.