text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Fix bug with intentional trailing spaces in instruct template prefix

Open netrunnereve opened this issue 10 months ago • 0 comments

Checklist:

If we have an intentional trailing space in an instruct template prefix (e.g. '[INST] ' + message['content'] + ' [/INST]') it gets stripped out during impersonation and we end up with something like [INST]<impersonated message>. However when I press generate it gets corrected back into [INST] <impersonated message>, which causes the backend to reprocess all the impersonated text.

netrunnereve avatar Apr 13 '24 04:04 netrunnereve