gorilla

gorilla copied to clipboard

[feature] functions is deprecated, use tools instead

1

**Is the feature request related to a problem?** OpenAI's modern client has deprecated the functions syntax. When using the recent 'tools' syntax, gorilla will only reply with generic text, e.g....

chris-aeviator

Added new Sequencing datasets: API BLEND EVAL

1

Dataset consists of SeqATIS, SeqSNIPS, SeqSGD, SeqMultiWOZ,SeqTopV2. Added two versions sequencing and sequencing lite with their Licenses . Added code changes pertaining to the two datasets. SeqTopV2 full dataset is...

sadhana01

[BFCL] Is this the same data used for the leaderboard?

3

hello. Thank you for making the BFCL public. I have a question. Is the data in “Berkeley-Feature-Currency-Leaderboard/Data” the same data you used for the leaderboard (https://gorilla.cs.berkeley.edu/leaderboard.html)? I'm curious if the...

hexists

Make BFCL User-Friendly and Easy to Extend

3

Thank you for open-sourcing BFCL and for your efforts in maintaining it. As I explored the codebase, I noticed some areas for improvement, including duplicate functions, constants, and variables that...

devanshamin

[BFCL] Adds support for parallel inference and batching

2

# Parallel Inference Support for berkeley-function-call-leaderboard This PR adds support for running `berkeley-function-call-leaderboard` inference in parallel, reducing running time by 4x or more depending on `--batch-size`. ## Changes ### Modifies...

TikZSZ

[bug] Hosted Gorilla: <Issue>

2

Exception: You tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API. You can run `openai migrate` to automatically...

francoisdevigne

Clarify Documentation About Running The Benchmark

1

I had trouble following [the instructions for running the benchmark](https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard/README.md) 1. Why do we have to apply the credentials one file at a time? Don't all the files get concatenated...

hamelsmu

Consistent Metric for OSS Model Cost

2

We need to be consistent in our metrics to determine the cost for OSS models. If a model is hosted locally and has `OSS_LATENCY`, then it should not belong to...

HuanzhiMao

The current BFCL leaderboard table is built using basic HTML, which has made it increasingly difficult to add new functionalities. To address this, the leaderboard table is overhauled to use...

HuanzhiMao

The mapping from test category name to test file path is repeated three times, which is bad. - `test_files` in `eval_data_compilation.py` - `test_categories` in `openfunctions_evaluation.py` - `TEST_CATEGORIES` in `model_handler/constant.py`

HuanzhiMao