AQLM icon indicating copy to clipboard operation
AQLM copied to clipboard

question about the finetune

Open LiMa-cas opened this issue 5 months ago • 6 comments

  1. is the finetune need each layer? could I used for some layers finetune once?
  2. is codebook quantized method is slower than AWQ?
  3. when I inference,it is successful when max_new_tokens=512, but failed when max_new_tokens=2048

LiMa-cas avatar Sep 23 '24 02:09 LiMa-cas