Ashmit Bhattarai comments

Repositories
Issues
Comments

Results 1 comments of


                                            Ashmit Bhattarai

Cannot merge LORA layers when the model is loaded in 8-bit mode

Anyone found a way to solve this without loading the model in 16bit? My GPU cannot load whole Falcon-40B (in 16-bit), using `device_map=auto` and `offload_folder` causes the python code to...