LoRA for fine tuning with InternVL3-5

[jbuchananr]

Issue

Hello, I currently use LoRA training to fine tune InternVL due to data set size constraints.

I ran into an issue with fine tuning 3.5 as internvl_chat_gpt_oss/internvl/model/internvl_chat/modeling_internvl_chat.py no longer as a LoRA implementation.

I attempted a modification based off the old implementation, but I am not sure if its correct, can anyone assist or already made the change?

Changes

Added back in wrap_llm_lora(...), wrap_backbone_lora(...) In wrap_llm_lora(...) had to modify for Qwen3

 elif self.llm_arch_name in ['Qwen3ForCausalLM', 'LlamaForCausalLM']:
            target_modules = ['self_attn.q_proj', 'self_attn.k_proj', 'self_attn.v_proj', 'self_attn.o_proj',
                              'mlp.gate_proj', 'mlp.down_proj', 'mlp.up_proj']

Modification in def forward(...)

model_to_search = self.language_model

if hasattr(model_to_search, "base_model"):
    model_to_search = model_to_search.base_model.model

if hasattr(model_to_search, "model"):
    layers = model_to_search.model.layers  # This is QWen3Model.layers
elif hasattr(model_to_search, "layers"):  # Fallback
    layers = model_to_search.layers
else:
    raise AttributeError("No layers attribute found in model")

for layer in layers:
.....

I was able to get this to run with a fine tuning script, but wanted to check with team first on if these changes are correct before continuing with my tuning and development. If this implementation, is correct I will also go ahead and submit a PR with the changes.

[paulpacaud]

Hi @Weiyun1025, do you plan to release a LoRA fine-tuning script for InternVL3.5 ?

[Arseny5]

Hi @Weiyun1025, do you plan to release a LoRA fine-tuning script for InternVL3.5 ?

Hello! I made a clean, reproducible notebook for fine-tuning InternVL-3.5 using multiple PEFT strategies (LoRA, QLoRA, and full-tuning). You can easily use it for fine-tuning on your tasks. Link: https://github.com/Arseny5/InternVL-3.5-QLoRA-Fine-tune