litgpt icon indicating copy to clipboard operation
litgpt copied to clipboard

Overwrite system prompt on load/generate

Open twsl opened this issue 1 year ago • 1 comments

Currently the prompt style is created on model generation and cannot easily be changed. However it would make sense to update the system prompt for each generation. to offer more flexibility. While the PromptStyle format is defined for each supported model architecture, there is no need to enforce the system prompt text itself. Therefore I suggest an extension to the PromptStyle class to update the system prompt

twsl avatar Dec 21 '24 00:12 twsl

Thanks for the suggestion. This is a reasonable extension. Unfortunately, I currently don't have the capacity to work on it, but maybe my colleagues can weigh in here.

rasbt avatar Jan 08 '25 22:01 rasbt