litgpt icon indicating copy to clipboard operation
litgpt copied to clipboard

Avoid the `convert_hf_checkpoint` step

Open carmocca opened this issue 2 years ago • 0 comments

https://github.com/Lightning-AI/lit-parrot/blob/main/scripts/convert_hf_checkpoint.py is a script that converts a list of *.bin files into a single checkpoint file: lit_model.pth.

This has the disadvantage of:

  • adds 1 extra step to get started
  • the checkpoint weights are now duplicated in the filesystem
  • it takes time and memory to convert.

This is particularly interesting for inference. For training/fine-tuning, the checkpoints generated will still be single file. We would need to support loading both options.

Instead, we could write a function lazy_load_from(checkpoint_dir) that does the weight mapping on the fly.

carmocca avatar Jun 15 '23 03:06 carmocca