litgpt
litgpt copied to clipboard
Avoid the `convert_hf_checkpoint` step
https://github.com/Lightning-AI/lit-parrot/blob/main/scripts/convert_hf_checkpoint.py is a script that converts a list of *.bin files into a single checkpoint file: lit_model.pth.
This has the disadvantage of:
- adds 1 extra step to get started
- the checkpoint weights are now duplicated in the filesystem
- it takes time and memory to convert.
This is particularly interesting for inference. For training/fine-tuning, the checkpoints generated will still be single file. We would need to support loading both options.
Instead, we could write a function lazy_load_from(checkpoint_dir) that does the weight mapping on the fly.