sanjay920

btw anthropic appears to be working in 1.15.3

Ideally from a PeftModel so I can convert like it's possible in llamacpp: https://github.com/ggerganov/llama.cpp/blob/master/convert-lora-to-ggml.py Or if one merges the lora adapter with the base model - so a [`GemmaModel`](https://huggingface.co/docs/transformers/main/en/model_doc/gemma#transformers.GemmaForCausalLM) to...

works fine in [v0.5.1](https://github.com/vllm-project/vllm/releases/tag/v0.5.1)