FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Modifying the answer format (### Human: xxx , ### Assistant: xxx)

Open rlancemartin opened this issue 2 years ago • 0 comments

I've been using Vicuna for Question-Answering.

I'm using the py-bindings (llama-cpp-python) and LangChain.

My prompt template is:

template = """Use the following pieces of context to answer the question at the end. Use three sentences maximum. 
{context}
Question: {question}
Answer: Think step by step """

I'm initializing the model:

llm = LlamaCpp(
                model_path="/<path>/vicuna_13B/ggml-vicuna-13b-4bit.bin",
                callback_manager=callback_manager,
                verbose=True,
                n_threads=6,
                n_ctx=2048,
                use_mlock=True)

I notice it will answer questions in a rhetorical style with ### Human: and ### Assistant: in the answer:

### Human: xxx? 
### Assistant: xxx

Have others seen this and / or should I be using an alternative prompt?

(Note: this question also exists here.)

rlancemartin avatar May 08 '23 16:05 rlancemartin