litgpt
litgpt copied to clipboard
Overwrite system prompt on load/generate
Currently the prompt style is created on model generation and cannot easily be changed.
However it would make sense to update the system prompt for each generation. to offer more flexibility.
While the PromptStyle format is defined for each supported model architecture, there is no need to enforce the system prompt text itself.
Therefore I suggest an extension to the PromptStyle class to update the system prompt
Thanks for the suggestion. This is a reasonable extension. Unfortunately, I currently don't have the capacity to work on it, but maybe my colleagues can weigh in here.