Sebastian Raschka
Sebastian Raschka
Sounds great, thanks. I will make a reminder to test this on Sunday/Monday!
How would you check it? Via the [nccl-tests](https://github.com/NVIDIA/nccl-tests) tool?
> No, nccl-tests are to measure performance. We don't need it. I agree. That would be overkill, which is why I implemented the current approach. > Anyway, I think your...
Hi there. It could be that you just need to add a blank `__init__.py` file. Please note that I am no longer developing or maintaining the code in this repo,...
Might be a LitData bug. Reported it here with a smaller self-contained example that doesn't use LitGPT: https://github.com/Lightning-AI/litdata/issues/367
It seems to be related to the MLP class: ## Has problem: - microsoft/phi-2 - GptNeoxMLP - EleutherAI/pythia-2.8b - GptNeoxMLP - stabilityai/stablelm-base-alpha-7b - GptNeoxMLP - google/gemma-2-2b - GemmaMLP ## Is...
Hi there, these are good questions. Off the top of my head, the major usage difference is the dataset. The `finetune_*` scripts are mainly designed for instruction-finetuning. (I wanted to...
That's a fair point, but there is this philosophy in this repo that some code duplication isn't bad if it helps with readability. Because too much refactoring and code sharing...
Closing to clean up the issues a bit. But please feel free to respond or reopen in case you have additional questions.
Thanks for reporting! The `--convert_checkpoint false` flag is not for the `.safetensor` file conversion but for the `HF -> LitGPT` conversion. Some models on the HF Hub have `.bin` files...