gorilla icon indicating copy to clipboard operation
gorilla copied to clipboard

[feature] Add multi-turn conversational function calling category for benchmarking

Open Pernekhan opened this issue 9 months ago • 2 comments

Is the feature request related to a problem? Currently, there are no benchmarking for multi-turn conversations.

Sometimes assistant needs to ask for more information before calling the functions. For example: User: Book me a flight to San Francisco? [functions: book_flight(from, to, date) Assistant: Tell me from where you're flying from and on what date? User: From London on May 25th 2024. Tool call: book_flight(from=London, to=San Francisco, date=2024-05-25) ...

Describe the solution you'd like

I'd like a new category to be added to the existing list. python openfunctions_evaluation.py --model MODEL_NAME --test-category multiturn

Additional context

Pernekhan avatar May 23 '24 14:05 Pernekhan