Benjamin Bossan

> I believe there are ways to locate the existing keys in a template then we can compare those against the provided template kwargs. Definitely something to look into That...

[New kernels](https://github.com/casper-hansen/AutoAWQ_kernels/releases/tag/v0.0.7) should solve this, right?

ping @sayakpaul maybe also interesting for diffusers

The accelerate issue you mentioned sounds very similar. Do you see the same error when using Q-LoRA (i.e. without DoRA)? Could you try downgrading accelerate and check if this resolves...

**Update**: DoRA and QDoRA training with FSDP should be fixed in #1806. If you install from the latest PEFT main, it should thus work. Please also check the PR description...

Thanks for the reviews. I'll wait a bit in case @iuliaturc has time to check if this PR fixes the initial issue. If we don't hear back, I'll merge in...

Thanks for the confirmation.

Can you show the code you used to initialize this adapter and to train it? Also, is there a specific reason why you use 0.6.0 or could you upgrade to...

> Below is my code： There is still some part missing, most notably what base model you use and how you load it. Please also share your `args.layers_ranks`. > how...

> 1.Layers_ranks is a list that defines the rank values ​​of different layers. I use this to define rank_pattern. Yes, I got that, but to be able to debug, I...