how to run VILA1.5-40B-AWQ

[chenxinhua]

Please provide a script to run the VILA1.5-40b int4 quantized model. like this:

[maoki109]

Not sure if this is exactly what you're looking for, but they have instructions and usage examples for running quantized VILA models here:

https://github.com/mit-han-lab/llm-awq/tree/main/tinychat

You will have to scroll to the bottom for the section regarding VILA models.