Kerry issues

Repositories
Issues
Comments

Results 2 issues of


                                            Kerry

a critical loss drop happen after each epoch ending

![image](https://github.com/artidoro/qlora/assets/54245159/e41a9a44-c67e-4c37-819b-ecb79e7ecea9) I am curious why the train loss drops after each epoch and tends to converge within one epoch. @artidoro The problem is that the train loss tends to always...

How to quantize models with Triton v2

It's fine to load a quantized model with triton_v2 https://github.com/AutoGPTQ/AutoGPTQ/blob/caf343b1826301c15f90e2e119cabd0347acfcdf/auto_gptq/modeling/_base.py#L689-L713 However, I noticed that there's currently no documentation or pull request detailing how to obtain a quantized model with the...