llama
llama copied to clipboard
meta-llama
Few Shot Learning in Chatbot manner?
Howdy, really appreciate your amazing work, and thank you for all the efforts that have been made.
I want to ask about some procedures for doing few-shot learning in the LLama2 chatbot setting. I am following the example provided in example_chat_completion.py and have some confusion about the manner.
I want to make sure that few-shot examples are in the following manner:
message = [
{"role": "user", "content": "Question 1 ..."},
{"role": "assistant", "content": "Answer 1 ..."},
{"role": "user", "content": "Question 2 ..."},
{"role": "assistant", "content": "Answer 2 ..."},
{"role": "user", "content": "Question 3 ..."},
{"role": "assistant", "content": "Answer 3 ..."},
...
{"role": "user", "content": "Question ..."},
]
If that is the case, I have another issue: if I want to do many-shot learning, the model will encounter: This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (4096). Depending on the model, you may observe exceptions, performance degradation, or nothing at all. The output from LLama2 also becomes mojibakes.
(Here, I assemble all examples in one dialog, and is that the reason for exceeding the maximum token limit? If that is, will splitting examples into multiple dialogs help mitigate this issue, but it is also kind of making the many-shot learning into multiple few-shot learnings?)
Do you have any suggestions on implementing a many-shot learning on the LLama2 chatbot? Grateful for any advice!
Hi @hxu105, can you share a bit more on what you are trying to accomplish here?
Hi @WuhanMonkey, my task is kind of difficult for LLMs, so I would like to add more examples in few-shot learning to see if LLMs can learn a correct pattern from human-level inference. My question is more about the formatting of few-shot learning. Many thanks!
Hi @hxu105, the 4096 max length limits regulate both input and output combined together. Likely your prompt is getting closer to that limit and output might hit length exception. The example_chat_completion.py should have the correct formatting of few-shot learning. You can also try adding the many-shot learning example in system prompt instead.