Andrei-Aksionov

Thanks for the response. `interval` parameter affects memory consumption for long running program quite a bit. I think such information should be reflected in the README file (not as Known...

Looks like a fairly popular model. HF reports over 1 million of downloads (last month). After #850 is merged, this can be the next goal. Though, it's better to be...

Hey Ivan, Thanks for the answer. > Any function that accepts and returns PyTorch tensors can be registered to work with Thunder. Sounds promising. Maybe the Readme file should reflect...

You are talking about Flash Attention lecture (haven't seen it yet)? If it's, so I think it would be a cool (and somewhat flashy) example.

I'll take a look at it tomorrow.

Hey @Dev-Khant The config seems to be ok, the only missing part was `rotary_percentage`, which needs to be `1.0`, since it's used in calculation of `rope_n_elem` (for RoPE embeddings): https://github.com/Lightning-AI/litgpt/blob/main/litgpt/config.py#L92...

These names ... I guess the next will be Nora 😆. ---- Thanks @rasbt It looks like it doesn't require a lot of changes, so I think it worth adding....

Thanks for the explanation 🤗. Now I see why it's implemented in that way. Cool, looks like we settled.

Only one test fails: > tests/test_config_hub.py::test_config_help[litgpt/pretrain.py-https://raw.githubusercontent.com/Lightning-AI/litgpt/main/config_hub/pretrain/tinystories.yaml] The reason is that in the main branch, the yaml file contains the old bias notation. After the PR is merged, this fail should...

> This also signals a breaking change. Can you add backwards-compatibility code to Config as we had in the past for other arguments? Sure. But before we handled this in...