llm-foundry icon indicating copy to clipboard operation
llm-foundry copied to clipboard

Set use_cache back to True for HF checkpointer

Open eldarkurtic opened this issue 1 year ago • 4 comments

Most HF models have use_cache set to True by default, which is manually changed to False in llm-foundry (most likely due to https://github.com/huggingface/transformers/issues/28056). This PR sets use_cache back to True before saving the model with the HF checkpointer.

This makes it a bit more convenient to use models trained with llm-foundry, without having to manually edit config.json and generation_config.json to set use_cache.

eldarkurtic avatar Aug 27 '24 08:08 eldarkurtic