VILA icon indicating copy to clipboard operation
VILA copied to clipboard

Published 20 hours ago • verified publisher icon NVlabs

LongVILA - compatibility with other LLMs

Open orrzohar opened this issue 1 year ago • 1 comments

Hi! very impressed by your work with LongVILA. I would like to do long context with Qwen/LLaMA3.1, but currently, i only see support for Mixtral.

Any chance you plan on releaseing long context support for these models soon?

Best, Orr

orrzohar avatar Aug 20 '24 03:08 orrzohar

Looking at the huggingface release -- it appears that the base LLM is LLaMA3.1? If yes -- could you please advise how this is implemented? I want to use this on my own model for long-context support, is this some attention implementation add-on?

orrzohar avatar Aug 20 '24 03:08 orrzohar

LongVILA models and docs have been released. Close for now. Feel free to reopen if further questions.

Lyken17 avatar Mar 09 '25 04:03 Lyken17