gorilla
gorilla copied to clipboard
[feature] Add multi-turn conversational function calling category for benchmarking
Is the feature request related to a problem? Currently, there are no benchmarking for multi-turn conversations.
Sometimes assistant needs to ask for more information before calling the functions. For example: User: Book me a flight to San Francisco? [functions: book_flight(from, to, date) Assistant: Tell me from where you're flying from and on what date? User: From London on May 25th 2024. Tool call: book_flight(from=London, to=San Francisco, date=2024-05-25) ...
Describe the solution you'd like
I'd like a new category to be added to the existing list.
python openfunctions_evaluation.py --model MODEL_NAME --test-category multiturn
Additional context