How to conduct full-tuning without LoRA?

[A11en0]

When I try to modify the original finetune.py script to conduct full tuning, which return an error like below:

I comment everything related to peft except model=prepare_model_for_int8_training(model)

and

    old_state_dict = model.state_dict
    model.state_dict = (
        lambda self, *_, **__: get_peft_model_state_dict(
            self, old_state_dict()
        )
    ).__get__(model, type(model))

[Oxi84]

I get the same, this is probably because you cant train on fp16 for some reason.

If you enable fp32 in the model (no lora) then it works, but it requires a lot of memory, i even get out of memory error on 4X 40 GB A100.

[adf1178]

Modify the load_in_8bit parameter as False can solve the problem

[A11en0]

I get the same, this is probably because you cant train on fp16 for some reason.

If you enable fp32 in the model (no lora) then it works, but it requires a lot of memory, i even get out of memory error on 4X 40 GB A100.

It's because LlamaForCausalLM do not achieve the Quantization function?