LAVIS icon indicating copy to clipboard operation
LAVIS copied to clipboard

Published 20 hours ago • verified publisher icon salesforce

Fine-tuning InstructBLIP?

Open alpayariyak opened this issue 1 year ago • 34 comments

alpayariyak avatar May 17 '23 14:05 alpayariyak

It seems that their fine-tuning strategy is similar to the standard training approach for VQA. I noticed that the blip2_vicuna_instruct.py file includes a predict_answers function, which is commonly used in VQA tasks.

To use their approach, you can prepare your datasets as they've described, including image, text_input, and text_output, and then launch train.py. However, I would also like to see more training details to better understand their methodology.

iamwangyabin avatar May 18 '23 06:05 iamwangyabin

It seems the run_scripts do not include anything related to InstuctBlip. Will the official release the code for pre-training and fine-tuning of InstructBlip?

chloejiang avatar May 22 '23 01:05 chloejiang

Bump, could you expand on the Instruction-Tuning section for the InstructBLIP model page? It's not clear how to do this or even if the full code necessary has been released.

austinmw avatar May 24 '23 21:05 austinmw

Could you release VQA finetuning script for instructBLIP?

edchengg avatar May 31 '23 18:05 edchengg

Could you release VQA finetuning script for instructBLIP?

meet the same problem

aopolin-lv avatar Jun 30 '23 19:06 aopolin-lv

same question

hangzeli08 avatar Jul 02 '23 09:07 hangzeli08

same question

qwqwq1445 avatar Jul 14 '23 07:07 qwqwq1445

same question,how to fine tune instruct blip?

tigerzjh avatar Aug 07 '23 12:08 tigerzjh

I would also like to know how to fine tune instruct blip

floriankark avatar Aug 11 '23 16:08 floriankark

Could the author provide the fine-tuning script of InstructBLIP? @LiJunnan1992

Richar-Du avatar Aug 14 '23 07:08 Richar-Du

mark!

Tower0823 avatar Aug 14 '23 09:08 Tower0823

mark

dydxdt avatar Aug 16 '23 10:08 dydxdt

mark

g2zr004 avatar Aug 22 '23 10:08 g2zr004

mark!

Oklahomawhore avatar Sep 11 '23 02:09 Oklahomawhore

same question here

sdc17 avatar Sep 23 '23 14:09 sdc17

mark

lxmcwt avatar Sep 26 '23 14:09 lxmcwt

same question

gwyong avatar Oct 01 '23 19:10 gwyong

+1

control-spiderman avatar Oct 06 '23 06:10 control-spiderman

+1, pretrain vicuna at stage2 also need to be modified

liu3xing3long avatar Oct 08 '23 03:10 liu3xing3long

mark

Lanyu0303 avatar Nov 27 '23 02:11 Lanyu0303

+1

him-mah10 avatar Nov 27 '23 11:11 him-mah10

mark

santaboi avatar Nov 29 '23 18:11 santaboi

mark

Clement25 avatar Dec 02 '23 16:12 Clement25

Guys, are there any training scripts for InstructBLIP on captioning (not VQA) tasks? Something like https://github.com/salesforce/LAVIS/blob/main/run_scripts/blip2/train/train_caption_coco.sh but for InstructBLIP?

Yuancheng-Xu avatar Dec 03 '23 22:12 Yuancheng-Xu

mark

owlsan49 avatar Dec 17 '23 02:12 owlsan49

mark

idor980 avatar Jan 07 '24 15:01 idor980

mark！

dszpr avatar Jan 22 '24 09:01 dszpr

https://github.com/AttentionX/InstructBLIP_PEFT?tab=readme-ov-file It semms like the authors have no motivatioon to release the finetuning script, but here is a repo that claims to do the same

findalexli avatar Feb 07 '24 06:02 findalexli

We have released the finetuning scripts, so let me know if you have any problem!

waitzkin avatar Feb 15 '24 07:02 waitzkin

We have released the finetuning scripts, so let me know if you have any problem!

Could you please tell me where it is? I found no finetuning scripts for instructblip under run_scripts/train/BLIP2

Clement25 avatar Feb 15 '24 11:02 Clement25