gpt-oss Training Fails (20B)

[linkspreed]

Hello,

we try to train the GPT OSS 20B with a Colab T4 GPU and receive this error:

AttributeError                            Traceback (most recent call last)
[/tmp/ipython-input-3842036555.py](https://localhost:8080/#) in <cell line: 0>()
      1 from trl import SFTConfig, SFTTrainer
      2 from transformers import DataCollatorForSeq2Seq
----> 3 trainer = SFTTrainer(
      4     model = model,
      5     tokenizer = tokenizer,

4 frames
[/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in __delattr__(self, name)
   2075             del self._modules[name]
   2076         else:
-> 2077             super().__delattr__(name)
   2078 
   2079     def _register_state_dict_hook(self, hook):

AttributeError: 'PeftModel' object has no attribute '_flag_for_generation'

Do you have any solution for that? Thank You very much!

[Yash-xoxo]

Model Loading Issue

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import get_peft_model, LoraConfig

# Load base model first
model = AutoModelForCausalLM.from_pretrained(
    "your-model-name",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Then apply PEFT configuration
peft_config = LoraConfig(
    task_type="CAUSAL_LM",
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
)

model = get_peft_model(model, peft_config)

pip install transformers==4.36.0 peft==0.7.0 trl==0.7.0

The most likely cause is a version mismatch between peft, transformers, and trl. This specific error has been reported with unsloth environments [Bug]PeftModelForCausalLM has not attribute '_flag_for_generation' · Issue #2490 · unslothai/unsloth, so updating to compatible versions should resolve the issue.

[ethicalvats]

facing the same issue

[IRONalways17]

/assign

[IRONalways17]

Issue Resolved - Ready for Merge Problem: AttributeError: 'PeftModel' object has no attribute '_flag_for_generation' when using GPT OSS 20B with SFTTrainer on Colab T4.

Solution: Added a simple compatibility fix that ensures the missing attribute exists before SFTTrainer initialization:

Compatibility fix for PEFT models with TRL SFTTrainer

if not hasattr(model, '_flag_for_generation'): model._flag_for_generation = False

Files Updated:

1. gpt-oss-(20B)-Fine-tuning.ipynb (Colab)
2. Kaggle-gpt-oss-(20B)-Fine-tuning.ipynb (Kaggle) Commit: c575535

This minimal, non-intrusive patch resolves the PEFT/TRL compatibility issue. Ready for merge into main branch.