langsmith-sdk icon indicating copy to clipboard operation
langsmith-sdk copied to clipboard

Issue: Add exception handling for nonexisting spits

Open jiayini1119 opened this issue 7 months ago • 2 comments

Issue you'd like to raise.

Hi LangChain Community,

I discovered that if I do

from langsmith import Client
from langsmith.evaluation import evaluate
client = Client()

and pass in data = client.list_examples(dataset_name = dataset_name, splits=[{"existing_split", "nonexisting_split"}]) ("existing_split" is a split that exists while "nonexisting_split" is a split that does not exist) in LangSmith's evaluate function, LangSmith will ignore the "nonexisting_split" and only evaluate on the "existing_split".

It would be better if error is raised for this case.

Thanks!

Suggestion:

No response

jiayini1119 avatar Jul 09 '24 04:07 jiayini1119