QWEN-Image / QWEN-Edit / HiDream / FLUX LoRA Training with Text_Encoder

[Dervlex]

Are we able to train for all aboves models also with text_encoder ? Because why ever when i set the "Clip_Strength" in Comfy to a higher value nothing happens. So i guess we are training currently "Model Only" LoRAs, correct ?

Thats completely in-efficent if you try to train a custom word / trigger word. I mean people are saying "Use Q5TeN" as trigger. But if the CLIP isnt trained, how should the LoRA effect then with a new trigger ?

Or do i get this wrong ?

[Anothergazz]

I have the same question. While trying to train a Lora of a person, it seems that all people of that gender are changed to the new person. I assume this is because the unique name of that person in the description is not being learnt by the model. Can some one confirm this is the case and how to get around this

[rzgarespo]

To determine if a LoRA for Qwen-Image (or related models like Qwen-Image-Edit) was trained on the text encoder (typically the Qwen2.5-VL-7B) in addition to the core model, you can inspect the LoRA's saved weights and configuration.

from safetensors.torch import load_file
import torch

# Path to your Qwen-Image LoRA file
lora_path = "path/to/your/qwen_image_lora.safetensors"

state_dict = load_file(lora_path)

# Common prefixes for text encoder LoRA keys in Qwen-Image setups
# These often include "text_encoder" or Qwen-VL layer targets like "model.layers", "q_proj", etc.
text_encoder_indicators = [
    "text_encoder.",
    "text_encoder1.",  # Sometimes for dual encoders
    "qwen_2.5_vl.",    # Qwen-specific VL text encoder prefix
    "model.layers.",   # Generic for Qwen-VL transformer layers if targeted
    "q_proj.",         # LoRA targets in attention layers of text encoder
    "v_proj.",
    "k_proj."
]

has_text_encoder_lora = False
for key in state_dict.keys():
    for indicator in text_encoder_indicators:
        if indicator in key.lower():
            has_text_encoder_lora = True
            print(f"Text encoder LoRA detected! Example key: {key}")
            break
    if has_text_encoder_lora:
        break

if not has_text_encoder_lora:
    print("No text encoder LoRA found—likely model-only training.")`

[Anothergazz]

Thanks for the feed back