text-generation-webui
text-generation-webui copied to clipboard
oobabooga
apply vicuna 1.1v instruction-following
vicuna 1.1v instruction is changed.
when change user name and Character's name, model more understand everything. because this form is the form when it is learned.
same page your can find this code
conv_vicuna_v1_1 = Conversation( system="A chat between a curious user and an artificial intelligence assistant. " "The assistant gives helpful, detailed, and polite answers to the user's questions.", roles=("USER", "ASSISTANT"), messages=(), offset=0, sep_style=SeparatorStyle.TWO, sep=" ", sep2="</s>", )
Yes, that's what I'm saying. You've set the roles correctly, but haven't set the context (aka system) to match that code. That code is written over two lines, but the string it defines is a single line.
There are other things we need to change to get this truly working. ~We need to stop it from using ###.~ (EDIT: No, that's in the character file, my mistake) We need to add the prompt in the prompts folder as a text file so it works in Notebook interface mode as well as Chat. And we need to automatically detect the v0 and v1.1 models and apply the instruct mode automatically.
I submitted a pull request to your branch with my improvements. But we still need to get automatic detection working.
Is it really that beneficial to use the exact context used in the training prompt?
Asking out of genuine curiosity.
In my anecdotal experience, I started out by always trying to match the context of my prompts with the context used by the model during training.
But eventually I got lazy and stopped doing that, and started using the same context across Alpaca/Vicuna/Koala/etc, and didn't notice any drop in quality. (Again, this is purely anecdotal and qualitative)
It makes sense that the model would perform better if you prompt it with a pattern that matches its training. But I do wonder how much it actually matters.
I don't know how much it matters. But it presumably helps at least a little, so we should do it correctly by default.
It is also important to note that Vicuna 1.1 uses a end_of_turn defined as </s> as described in https://github.com/lm-sys/FastChat/blob/main/docs/vicuna_weights_version.md#example-prompt-weight-v11
This end of turn only appears after the Assistant's reply, which is currently not supported by this repo. Maybe introducing a role-specific separator might be useful.
USER: Hello!
ASSISTANT: Hello!</s>
USER: How are you?
ASSISTANT: I am good.</s>
This PR implements the exact prompts https://github.com/oobabooga/text-generation-webui/commit/a777c058aff905e142700b912cd8e3ddbd3fe8e1
I couldn't find much documentation on Vicuna. In particular, a space after ### Assistant: or ASSISTANT: seems necessary according to a README that I found, but the model only generates output without that space: https://github.com/oobabooga/text-generation-webui/pull/1579
If someone can find an improvement to the templates, please submit a PR. They can be found here: https://github.com/oobabooga/text-generation-webui/tree/main/characters/instruction-following
This PR implements the exact prompts a777c05
It's wrong. That's not what the context string is supposed to be.
If someone can find an improvement to the templates, please submit a PR. They can be found here: https://github.com/oobabooga/text-generation-webui/tree/main/characters/instruction-following
I'm working on it. Will submit a new PR soon.