llama-stack-apps icon indicating copy to clipboard operation
llama-stack-apps copied to clipboard

Clarification on the system prompt for custom tool use

Open ricklamers opened this issue 6 months ago • 15 comments

Awesome work! Just a quick question about the correct system prompt:

in the docs https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1#user-defined-custom-tool-calling this is used:

If a you choose to call a function ONLY reply in the following format:
<{start_tag}={function_name}>{parameters}{end_tag}
where

start_tag => `<function`
parameters => a JSON dict with the function argument name as key and function argument value as value.
end_tag => `</function>`

Here is an example,
<function=example_function_name>{"example_name": "example_value"}</function>

Reminder:
- Function calls MUST follow the specified format
- Required parameters MUST be specified
- Only call one function at a time
- Put the entire function call reply on one line"
- Always add your sources when using search results to answer the user query

You are a helpful Assistant.

While in the repo this is used:

Think very carefully before calling functions.
If you choose to call a function ONLY reply in the following format with no prefix or suffix:

<function=example_function_name>{{"example_name": "example_value"}}</function>

Reminder:
- If looking for real time information use relevant functions before falling back to brave_search
- Function calls MUST follow the specified format, start with <function= and end with </function>
- Required parameters MUST be specified
- Only call one function at a time
- Put the entire function call reply on one line

Furthermore, could you clarify if the "Only call one function at a time" implies parallel tool use is not intended to be used for these instruction tuned models (Llama 3.1 family)?

e.g. "Please get the weather for San Francisco and Tokyo" can't generate:

<|start_header_id|>assistant<|end_header_id|>

<function=get_weather>{"location": "San Francisco"}</function>
<function=get_weather>{"location": "Tokyo"}</function><|eot_id|>

Thanks for clarifying!

Rick Lamers AI Researcher at Groq

ricklamers avatar Aug 02 '24 11:08 ricklamers