Feature: QDoRA

[zaithottakath]

• Made the DoRALinear layer work with QuantizedLinear
• Tests for DoRALinear
• bug fix for recalculating the magnitude (self.m) when changing the linear layer of the DoRALinear instance

[zaithottakath]

Fine tuning Llama 3 with 1 layer and the default 2 keys using DoRA:

 Loading configuration file adapter_config.json
Loading pretrained model
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading datasets
Training
Trainable parameters: 0.001% (0.112M/8030.261M)
Starting training..., iters: 112164
Iter 1: Val loss 2.108, Val took 77.962s
Iter 10: Train loss 2.875, Learning Rate 2.000e-05, It/sec 0.460, Tokens/sec 478.967, Trained Tokens 10402, Peak mem 17.474 GB
Iter 20: Train loss 2.898, Learning Rate 2.000e-05, It/sec 0.488, Tokens/sec 447.622, Trained Tokens 19583, Peak mem 17.474 GB
Iter 30: Train loss 2.511, Learning Rate 2.000e-05, It/sec 0.428, Tokens/sec 385.508, Trained Tokens 28584, Peak mem 21.233 GB
Iter 40: Train loss 2.101, Learning Rate 2.000e-05, It/sec 0.262, Tokens/sec 316.188, Trained Tokens 40666, Peak mem 21.359 GB
Iter 50: Train loss 2.020, Learning Rate 2.000e-05, It/sec 0.211, Tokens/sec 295.234, Trained Tokens 54680, Peak mem 26.664 GB
Iter 60: Train loss 1.928, Learning Rate 2.000e-05, It/sec 0.243, Tokens/sec 317.582, Trained Tokens 67724, Peak mem 26.664 GB
Iter 70: Train loss 1.807, Learning Rate 2.000e-05, It/sec 0.250, Tokens/sec 336.838, Trained Tokens 81204, Peak mem 26.664 GB
Iter 80: Train loss 1.754, Learning Rate 2.000e-05, It/sec 0.252, Tokens/sec 342.972, Trained Tokens 94793, Peak mem 26.664 GB
Iter 90: Train loss 1.636, Learning Rate 2.000e-05, It/sec 0.199, Tokens/sec 340.777, Trained Tokens 111876, Peak mem 26.664 GB

[zaithottakath]

Hi @awni , I was wondering if there's any feedback or additional information needed for this contribution? I'm eager to help improve the library and am happy to make any necessary adjustments.

[awni]

Sorry for the delay! Will plan to review this and share feedback shortly.

[awni]

This is a nice addition! Apologies for the delayed review.

Could you rebase and address the comments and then we can merge it? Thank you!