Murtaza Nasir comments

Repositories
Issues
Comments

Results 2 comments of


                                            Murtaza Nasir

[BUG] Can not save quantized model to disk: "you shouldn't move a model that is dispatched using accelerate hooks."

Were you able to fix this? I am getting the same error. I am using the example script `quant_with_alpaca.py`, and it looks like the model gets quantized. It even tests...

[BUG] Following the quant_with_alpaca.py example but keep getting "You shouldn't move a model that is dispatched using accelerate hooks." and the model is never saved.

> same issue. Have you solved the problem? I haven't unfortunately. Please do share if you find a solution!