Bernsai

I am having the same issue with a A100 80GB: ```bash openllm start llama --model_id meta-llama/Llama-2-13b-chat-hf --quantize int4 ``` The error does not occur with the standard config of: ```bash...

Are there any updates on this issue @aarnphm? Thanks! Any workaround available?

Version `0.3.5` fixed that issue on my side. Thanks a lot for your support!