univa-JASON
univa-JASON
Thank you for your achievements. However, when image-text pair data and text-only data were included in the same batch, the following error occurred when running the code. ''' Traceback (most...
Thanks for your fast reply, but i got same error.. here is my compute loss code in local, maybe it is old version. ``` def compute_loss(self, model, inputs, return_outputs=False): if...
oh, that's ok. thank you so much for your help.
``` def get_vllm_embedding(self, data): if 'vision_hidden_states' not in data: dtype = self.vpm.embeddings.position_embedding.weight.dtype device = self.vpm.embeddings.position_embedding.weight.device tgt_sizes = data['tgt_sizes'] pixel_values_list = data['pixel_values'] vision_hidden_states = [] all_pixel_values = [] img_cnt = []...
1 text-image pair data and 1 text-only data in my sample test, it seems tgt_sizes = torch.tensor([]) in text-only data. but An error occurred because the data entered the ```if...
thanks a lot for your feedback! i modified the code but sadly got the same error.
22 how can i finetune this model with Text-only data and Image-Text data in same dataset?
oh, thanks for your answer. then only v2_6 support multi image? And when finetune with interleaved dataset, that format can be used for v2_6?
i see. thank you so much!!
thank you for your answer! i have 1 more question, Can I apply wsd scheduler in This repo's finetuning code?