DeepSpeed error with LoRa

[g-batalhao-a]

Hi,

I was trying to use the fine-tune script with a quantized Mistral and I get the following error: ValueError: DeepSpeed Zero-3 is not compatible with "low_cpu_mem_usage=True" or with passing a "device_map".

After looking at this issue, I removed this line but then the error that appears is the following: RuntimeError: Only Tensors of floating point and complex dtype can require gradients

Could I get help solving this issue? Thanks