taylor-chaos comments

Repositories
Issues
Comments

Results 2 comments of


                                            taylor-chaos

Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu

same issue. quantization is all float8 default, and turn on Low VRAM and Layer Offloading(both 100%)

Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu

> I'm currently getting this with Qwen on the most current build using Auto Memory > > ``` > File "/home/me/ai-toolkit/run.py", line 120, in > File "/home/me/ai-toolkit/run.py", line 120, in...