donghong1

"Clarification on Multimodal Model Quantization and Default Calibration Dataset"

1

Hello, I have a few questions regarding the quantization of multimodal models: 1、Does the current version of AutoAWQ quantize only the language model, or does it also include the vision...

Hello, I encountered the issue while quantizing the llava-7b model. An error occurred: AttributeError: 'NoneType' object has no attribute 'shape'. My versions are autoawq==0.2.8, tokenizers==0.21.0, and transformers==4.47.1.How can I to...

I encounter some difficulties calculating the flops of multimodal large models like Llava.

Inquiry About Support for Evaluating AWQ Quantized Models

1

Hi, Thank you for your great work on this repository! I am currently exploring the use of AWQ for model quantization and was wondering if the repository supports evaluating models...