Billy Cao
Hi how do i get the last_logits out? i dont need logprob for every token, but just the last one. I am using llava1.6 mistral 7b and it has a...
@iceman-p Hi how did you load the 7b one? I am having trouble loading as i get https://github.com/haotian-liu/LLaVA/issues/1112
i didnt observe this using the chair example. Try deleting the pad related additions? i actually dont have a concrete evidence saying pad is even necessary.
Yea pad seem to be extra as they use unk as pad, so i guess should delete the pad related entries and set pad token id in various files to...
how did you train? i thought the training codes are not out?
> If I may ask, are your inference results good? I am trying to infer but seems the generated output ends weirdly. I use a single A-100 to generate an...
Well I cant do much about OOM. But I thought the repo said training code to be released...so you just used transformers trainer and it worked?
> The whole time, this issue was a lack of VRAM. > > Llava 1.6 takes multiple (overlapping) patches of the input image and uses them as input. This seems...
No i loaded in fp16 for inference, using sglang. Its very slow though and i think its silently offloading some to cpu ram
I have 32GB and its using more than 10GB