[Docs] Misleading documentation of the finetuning process

[paulpacaud]

📚 The doc issue

In the documentation "Enhancing InternVL2 on COCO Caption Using LoRA Fine-Tuning", it is written at the end to copy the config.json file from the original InternVL2-2B directory to the new merged model directory. The original config.json does not necessarily match the new config.json, especially, the parameter max_dynamic_patch can be different between the two configs. It means that if you finetune your model with let's say max_dynamic_patch=6 (as in the default shell scripts provided), and then you overwrite your config.json with the original model (that has max_dynamic_patch=12), you introduce a subtle error.

Suggest a potential alternative/fix

I am asking for clarifications from the authors or developers of this repository regarding this matter:

• Why do we need to copy this config.json file while there is already one in the folder of the fine-tuned model?

[wzczc]

the same question