vllm icon indicating copy to clipboard operation
vllm copied to clipboard

Published 20 hours ago • verified publisher icon vllm-project

[Feature]: How to run the int4 quantized version of the gemma2-27b model

Open maxin9966 opened this issue 6 months ago • 5 comments

🚀 The feature, motivation and pitch

How to run the int4 quantized version of the gemma2-27b model

Alternatives

No response

Additional context

No response

maxin9966 avatar Aug 04 '24 13:08 maxin9966