Iven2132
Iven2132
> Hello, I estimate that the effect of replacing the system message with this model is not good because a fixed system message was used during training, rather than training...
Can you give me example of doing that after making a Docker image? What to do after that? To run on my own infra On Mon, 29 Apr 2024, 21:05...
> Hi @Iven2132 it's not super streamlined, but the way to do this would be to use: > > ``` > $ truss image build > ``` > > to...
> Why do you need to use a smaller value? Hi, @merrymercy I should increase or decrease --mem-fraction-static? I'm running the Qwen 72b model on 4 * A100s 80GB. I'm...
> Thank you for the clarification with the formulas. I better understand now the issue. > > However I am still a bit puzzled and to me the guidance seems...
Did anyone get any solution? I'm getting same error with llava-next-72b
> Yes. They are supported. @lvhan028 So I can use the LLaVA-NeXT Qwen1.5-72B, LLaMA3-8B?
> Sorry, my bad. It probably needs to make some changes like PR #1579 does. I didn't find the checkpoints. Could you share the huggingface repo_id? lmms-lab/llama3-llava-next-8b: https://huggingface.co/lmms-lab/llama3-llava-next-8b lmms-lab/llava-next-72b: https://huggingface.co/lmms-lab/llava-next-72b...
> Do you know what minimum GPU memory requirements would be to serve these VL models? Thanks! A100s are best if you want you can run these on Modal, they...
> > > Do you know what minimum GPU memory requirements would be to serve these VL models? Thanks! > > > > > > A100s are best if you...