opik icon indicating copy to clipboard operation
opik copied to clipboard

[Bug]: Experiment Result Uploading Issue

Open JoesSattes opened this issue 2 months ago • 1 comments

What component(s) are affected?

  • [x] Opik Python SDK
  • [ ] Opik Typescript SDK
  • [ ] Opik Agent Optimizer SDK
  • [ ] Opik UI
  • [ ] Opik Server
  • [ ] Documentation

Opik version

Opik 1.9.14

Describe the problem

Hi Opik Team,

I'm running an opik.evaluate() experiment, and I'm seeing a strange issue.

The experiment runs successfully in my local console. I can see the final metrics and the "Uploading results to Opik..." message.

Here's my console output:

{'Evaluation_Metrics': {'A_Factual_Accuracy_Score': 9, 'B_Completeness_Score': 9, 'C_Conciseness_Formatting_Score': 10}, 'Overall_Score': 9, 'Detailed_Reasoning': "The response accurately denies any intent to 'wipe the world' and offers helpful assistance with home-related and useful life tasks,"}
╭─ Smart: General Test (1 samples) ──────╮
│                                             │
│ Total time:         00:00:03                │
│ Number of samples: 1                        │
│                                             │
│ BS-GE: Overall Score: 9.0000 (avg)          │
│ BS-GE: Factual Accuracy Score: 9.0000 (avg) │
│ BS-GE: Completeness: 9.0000 (avg)           │
│ BS-GE: Conciseness: 10.0000 (avg)           │
│                                             │
╰─────────────────────────────────────────────╯
Uploading results to Opik ... 
View the results link.

Here this my running script:

import os
from opik.evaluation import evaluate

from opik import Opik

client = Opik(
    api_key=<API>,
    project_name=<PROJ>,
    workspace=<ABC>)

dataset = client.get_or_create_dataset(name="Book Smart: General Test")

def evaluation_task(dataset_item):

    result = {"output": {
        "User_Query": dataset_item["User_Query"],
        "Agent_Response": dataset_item["Agent_Response"],
        "Ground_Truth": dataset_item["Ground_Truth"],
        "Knowledge_Category": dataset_item["Knowledge_Category"]
    }}

    return result


eval_results = evaluate(
experiment_name="smart-general-skill-evaluation",
  dataset=dataset,
  task=evaluation_task,
  scoring_metrics=[GeneralSkillMetric()],
  project_name="Smart: General Skill",
  nb_samples=1
)

The Problem: When I click the "View the results link", the experiment page in Opik is empty. It shows no traces or details, and it displays a message that the "dataset is deleted". This is incorrect—my dataset (Smart: General Test) is not deleted. I can still see it in my "Datasets" tab. Why is the experiment link showing no results and a "deleted dataset" message, even though the run was successful and the dataset still exists?

Image

Thanks for your help!

Reproduction steps and code snippets

No response

Error logs or stack trace

No response

Healthcheck results

No response

JoesSattes avatar Nov 15 '25 19:11 JoesSattes

@JoesSattes,

It looks like there might be a bug when dataset names contain colons (:). The SDK sends the name correctly, but when the experiment page loads, the backend might be failing to look up the dataset by name, causing the "dataset is deleted" message even though the dataset exists.

Workaround: Avoid colons in dataset names. Use:

  • "Book Smart - General Test" (hyphen)
  • "Book Smart General Test" (space)
  • "Book Smart_General Test" (underscore)

Can you try a workaround and see if that works?

dsblank avatar Nov 20 '25 18:11 dsblank

Thank you for the suggestion. This 'Book Smart General Test' (space) format works perfectly for me.

JoesSattes avatar Dec 06 '25 08:12 JoesSattes