InternVL

InternVL copied to clipboard

### Checklist - [X] 1. I have searched related issues but cannot get the expected help. - [X] 2. The bug has not been fixed in the latest version. -...

14H034160212

### Motivation Right now it appears that InternVL only supports SFT, but it would be helpful to expand on this with preference datasets. This would allow for an even more...

fabriceyhc

[Feature] Utilize the Internvl 2.0 for image-text retrieval

1

### Motivation thanks for your excellent work. I have found that in the internvl-g, we can find the retrieval code, which can be found also in clip benchmark. I wonder...

Hoantrbl

[Feature] Implement InternVL to llama.cpp

22

### Motivation Many llama.cpp users are requesting this so far. Ollama is one of the interfaces of llama.cpp and it is quite popular. Implementing it will significantly accelerate InterVL adoption...

James4Ever0

[Feature] Integration Of Custom Vision-Model without altering modeling_internvl_chat.py

2

### Motivation Task specific vision-models would perform better in that task rather than general purpose vision-model. So it would be better if we can simple pass our vision_model in the...

hamza-dev-12

ocr data label

5

hi，想请教下ocr data在预训练和sft阶段的具体label是怎样产生的？ 看了前面很多问题提到ocr的监督为'\ntext1\ntext2\ntext3', 比如[#536](https://github.com/OpenGVLab/InternVL/issues/536)、[#49](https://github.com/OpenGVLab/InternVL/issues/49),但是都没有提到如何组织顺序的。 是按照从左到右从上到下的启发式规则进行排序还是通过模型构建具体的顺序。 启发式规则在遇到一些奇怪结构的时候容易打乱语序，这样的监督是否反而会损害模型的性能哇？ 第二个就是看前面[#239](https://github.com/OpenGVLab/InternVL/issues/239)提到有部分带坐标框的ocr训练数据，想请教下带框ocr和不带框ocr的数据比例方便透露么？ 非常感谢!

xiaohuicomeon

[Bug] 断点二次训练merge lora失败

4

### Checklist - [X] 1. I have searched related issues but cannot get the expected help. - [ ] 2. The bug has not been fixed in the latest version....

linssonSUSUSU

### Motivation Hi gang, Im new to VLMs. I wonder how do you guys perform prompt tuning of InternVL2? PEFT PromptTuningConfig? Thx! ### Related resources _No response_ ### Additional context...

myc9799

llll111111

[Bug] ValueError: Unrecognized configuration class <class 'transformers_modules.InternVL-Chat-V1-5.configuration_internvl_chat.InternVLChatConfig'> to build an AutoTokenizer.

2

### Checklist - [X] 1. I have searched related issues but cannot get the expected help. - [X] 2. The bug has not been fixed in the latest version. -...

llll111111