Sebastian Raschka
Sebastian Raschka
I had no idea. Yeah, then let's revive @Andrei-Aksionov's #878
Glad to hear that the package has been useful to you in the past! Arg, and it sounds frustrating that these servers are not active anymore but I guess a...
We can close this as duplicate.
We also have a new documentation here: https://github.com/Lightning-AI/litgpt/blob/main/tutorials/pretrain.md which might be helpful. Please don't hesitate to reach out if you have additional questions or need help.
> The bins/safetensors exists but the lit_model.pth doesn't I'd say 1. Don't download but just print a message: "Skipping download. Files already exist. Override with --force_download true" 2. Then run...
Thanks for your suggestion and offering to contribute! Like you said, I think this could be useful indeed. We could probably add it via the --python_installation flag. Only supporting Linux...
I'll try to expand on that, maybe it could be based on LitServe
This was addressed in #1299
Flash attention now supports Alibi btw, so in that case, Alibi support in Lit-GPT could be a bit easier.
I think we should update this perhaps: