langsmith-sdk
langsmith-sdk copied to clipboard
Issue: Add exception handling for nonexisting spits
Issue you'd like to raise.
Hi LangChain Community,
I discovered that if I do
from langsmith import Client
from langsmith.evaluation import evaluate
client = Client()
and pass in
data = client.list_examples(dataset_name = dataset_name, splits=[{"existing_split", "nonexisting_split"}])
("existing_split" is a split that exists while "nonexisting_split" is a split that does not exist) in LangSmith's evaluate
function, LangSmith will ignore the "nonexisting_split" and only evaluate on the "existing_split".
It would be better if error is raised for this case.
Thanks!
Suggestion:
No response