Jee Jee Li
Jee Jee Li
> > @robertgshaw2-neuralmagic @comaniac There is a potential risk of illegal memory access, I have made changes but have not yet submitted them. Please refer to:[add_device_gurad](https://github.com/jeejeelee/vllm/blob/fix-moe-kernel/csrc/moe_align_block_size_kernels.cu#L115) > > Interesting. Do...
Could you provide your LoRA configuration? I might be able to implement this quickly
I will. can you provibe `target_modules` info?
Currently, vllm does not support LoRA inference for the visual encoder and projector components. It only supports LoRA inference for language models. Even if I were to implement support lorafor...
> deploying using merged lora is affecting performance How many LoRAs do you have? If you only have one, merging the weights would lead to higher inference efficiency.
@AkshataABhat II have preliminarily completed the work of supporting LoRA in InternVL, but note that it can still only add LoRA to the language model. You can try it out....
> > @AkshataABhat II have preliminarily completed the work of supporting LoRA in InternVL, but note that it can still only add LoRA to the language model. You can try...
> > > > @AkshataABhat II have preliminarily completed the work of supporting LoRA in InternVL, but note that it can still only add LoRA to the language model. You...