Santhana Mikhail Antony S
Santhana Mikhail Antony S
From my experience, I have found that AutoGPTQ does not build against Python 3.12.2. For me, both Python 3.10.12 and Python 3.11.8 works.
> Hi @hinthornw, > > What if I want to update the state values with the tool call? Is there a way to do it? My understanding was if a...
> Is it possible to modify multiple state value using a single node/function? Yes, it is.
> This is definitely on our roadmap and will be tackled in the coming weeks. Here are the priorities right now: > > 1. re-write the scheduling code and cache...
Does the project currently have any logging?
> It looks like there's a problem with your LoRA weights or you've loaded incorrect weights > > ```python > 2024-10-16T03:44:47.190186093Z ERROR 10-15 20:44:47 engine.py:160] raise ValueError(f"{name} is unsupported LoRA...
Does vLLM not support QLORA for GPTQ models yet?
> > Does vLLM not support QLORA for GPTQ models yet? > > vLLM does support QLORA for GPTQ models. If the loading of adapter works with Transformers, then it...
I think mine is a duplicate of #4186 . Looks like #8082 will fix it.
> You mean your lora config includes `modules_to_save`, right? If that's the case, the current vllm doesn't support this Yes, Sir. Am I right in saying 8082 will fix it?...