Convert lit-parrot model format back to huggingface format

[jharvill23]

Finetuning with lit-parrot is simple, but the generation capabilities are limited compared to huggingface. Could you please add support for converting the trained lit-parrot model back to the huggingface format so I can train with lit-parrot and generate with huggingface? I need to use beam search.

[lynngao]

Totally agree. I spent lots of time to find solutions but nothing works.

[carmocca]

There's two ways to do this. Either having the opposite operations of https://github.com/Lightning-AI/lit-gpt/blob/main/scripts/convert_hf_checkpoint.py#L19-L169 for each of the HuggingFace classes, or creating a HF Transformer model version of lit_gpt.model.

The former is simpler because the weights are loaded back into the original HF models. The latter is more powerful in that it could support the LoRA and Adapter modifications that we support in this repo

[lantiga]

I'd go with the first for the time being (@JustinGoheen volunteered).

The second is actually interesting. I guess it comes down to implementing a few classes as done in https://huggingface.co/tiiuae/falcon-7b/blob/main/modelling_RW.py (where RW stands for RefinedWeb, for those wondering :-) ). Let's keep it in mind for the future.

[thanhnew2001]

Any progress on this please. Highly appreciated.

[jxtngx]

@thanhnew2001 I'm starting a PR this week. I'll link it here when I submit the draft.

[Amshaker]

Hi @JustinGoheen ,

Is there any update regarding this PR?

Your progress in this matter is highly appreciated.

[Amshaker]

Hi @JustinGoheen @carmocca @lantiga ,

Is there any update regarding releasing the script to convert lit-gpt models to HuggingFace classes?

Your response is highly appreciated.

[jxtngx]

Here's my dev branch for this. I'll create the draft PR tomorrow (July 17) after I have implemented one or both of (copy_weights_gpt_neox, copy_weights_open_llama).

https://github.com/JustinGoheen/lit-gpt/tree/justin/hf-model-conversion

You can clone my branch with:

git clone justin/hf-model-conversion https://github.com/JustinGoheen/lit-gpt.git

Then download and convert the Falcon 7B weights, and mock the conversion back to HF with

python scripts/convert_lit_checkpoint.py

and then carry out a test with:

Note

pytest must be installed in your virtual environment

pytest tests/test_convert_lit_checkpoint.py