llama.cpp
llama.cpp copied to clipboard
Create "instruct" example
Currently, the main
example has a instruct
parameter which enables something similar to instruction-based mode. I haven't understood it completely, but this seems to be what the Alpaca models are created for.
Since we now support infinite generation (https://github.com/ggerganov/llama.cpp/issues/71#issuecomment-1483907574) it would be very useful to make a separate app that utilizes the new --keep
argument to create a question-answering bot that never stops. The tricky part is to keep the correct instruction prompt and "inject" the few-shot examples correctly, or whatever.
The main logic for context swapping / context rotation is here:
https://github.com/ggerganov/llama.cpp/blob/c2b25b6912662d2637d9c6e6df3a5de931e0d7ce/examples/main/main.cpp#L297-L324
Uncomment the printf
to help debug. Something similar will be needed in the new instruct
example.
Implementing this task will also help simplify the main
example as it will no longer need to support the --instruct
argument.