langchain-benchmarks

langchain-benchmarks copied to clipboard

Whenever we're ready with tool calling ```python # ("fireworks-firefunction-v1", ChatFireworks(model="accounts/fireworks/models/firefunction-v1", temperature=0)), # ("cohere-command-light", ChatCohere(temperature=0, model="command-light")), # ("cohere-command", ChatCohere(temperature=0, model="command")), # ("cohere-command-r", ChatCohere(temperature=0, model="command-r")), # ("cohere-command-r-plus", ChatCohere(temperature=0, model="command-r-plus")), # ("mistral-large-2402", ChatMistralAI(model="mistral-large-2402",...

eyurtsev

add notebook for custom chain benchmarking on extraction data

1

hwchase17

ConnectionError

3

Hi, I'm trying to run [custom_agent.py](http://localhost:8888/edit/Downloads/langchain-benchmarks-main/csv-qa/custom_agent.py) on my computer, when it comes to this line of code: `chain_results = run_on_dataset( client, dataset_name="Titanic CSV Data", llm_or_chain_factory=get_chain, evaluation=eval_config, )` it generates an...

nxby

I see here that code is using the GPT 4 model for the evaluation, since it its the most expensive model out there to run, is it possible to change...

daniellefranca96

eyurtsev

rlancemartin

Common question: I'm fine-tuning for an agent. What split of data should I prioritize collecting, and in what mixture? - Fewer long trajectories? - More short trajectories / single-step function...

hinthornw

Implementation of anthropic agent

1

WIP implementing using our own primitives

eyurtsev

Additional models

eyurtsev

Use `type` rather than `Type`

1

IMO we should use python conventions. If we think this is confusing, we can lower-case the column names in all html tables.

eyurtsev