Sebastian Raschka

Yes, good news is that it's possible. Generally, it works the same way as outlined in finetuning with LoRA howto: https://github.com/Lightning-AI/lit-llama/blob/main/howto/finetune_lora.md By default, when you run `python finetune/lora.py` it uses...

I don't now for sure, but it worked for me on 8 A100 cards

Actually, the `GPT` here is just a Python class. All other LLMs are very similar to GPT, which is why they are derived from that class in code. To choose...

I hope this helped. Please feel free to reopen if you have a follow up question.

I wonder perhaps if it is related to the tokenizer? It could also be a limitation of the terminal outputting certain characters. Unfortunately, I am not super familiar with working...

Thanks for your interest in this. This was requested by a colleague for a downstream application, but I haven't had a chance to think this through in terms of how...

Your jsonargparse example has been super helpful for understanding things a bit more @carmocca . Many thanks for this! But maybe it's because it's Fri evening but my brain is...

Awesome, thanks so much, this was great help! Figured it out now and got it to work. Many thanks, again learned something new!

I now got it to work as follows: ```bash litgpt finetune full \ --checkpoint_dir checkpoints/EleutherAI/pythia-160m # Specify optimizer and optimizer args: litgpt finetune full \ --checkpoint_dir checkpoints/EleutherAI/pythia-160m \ --optimizer torch.optim.SGD...

The only caveat now is that the class path still needs to be specified. I.e., only specifying the learning rate doesn't work ``` litgpt finetune full --optimizer.lr 200 --checkpoint_dir checkpoints/EleutherAI/pythia-160m...