Sebastian Raschka comments

Results 821 comments of


                                            Sebastian Raschka

OptimizerArgs

I hope this is ready now @carmocca

OptimizerArgs

It does. Let me investigate ...

OptimizerArgs

Should be fixed for good now @carmocca . I can switch the link to the original tinystories now that you have seen the green checks haha 😆

LR scheduler can result in a division by 0

Thanks for the note!

prompt_style

I think the easiest way here would be to use on of the existing datasets as templates. I remember that deita had multi turn questions in the dataset, so I...

prompt_style

Glad to hear you found it useful! There are currently so many todos, but yeah, adding DoRA to LitGPT some time would be great.

Remove old and unused LLMs

Yes, it should be relatively beginner friendly. Feel free to work on it.

Add resume for adapter_v2, enable continued finetuning for adapter

Thanks for looking into this. Sorry, I haven't spent much time on thinking through the ramifications here. But would the simple resuming from the `full` finetuning code not work in...

Conversion to HF checkpoint should generate a checkpoint format that can be loaded directly

Yes I agree, there were a few people on Discord recently struggling with this

Pretraining example from readme fails in Colab

Hi there, I was just running this code on a A10G in a Lightning Studio, and it worked fine. Do you have the error message you got or more explanation...