[FR]: Add total experiment cost

[Eduardo-Laquintana-Alonso]

Proposal summary

I am proposing to add experiment costs, which could include

1. A sum of the cost of each trace in an experiment
2. The average cost of a trace in the experiment.

Both values could be shown as columns in this view:

Inside of the project you can see the average cost of each trace in the project, and within the metrics of a project we can also see the total daily cost of the project, however there's no tracking back to a single experiment (see motivation). We can see avg. trace cost in the project view, but that will aggregate traces from different experiments, rendering the avg. less helpful.

Motivation

We treat experiments as independent runs of each of our application's prompts/tasks. As we balance comprehensiveness of a test (i.e., varied inputs) with cost, it would be useful to know how much a single prompt/task experiment costs over time, to be able to prioritize and optimize accordingly.

[juanferrub]

¡Hola, Eduardo!

Thanks for sharing this suggestion! I think it’s a great idea, and we really appreciate you taking the time to provide this feedback. We’ll review it internally and assess how it can fit into our roadmap.

Thanks for contributing to the project!

[Eduardo-Laquintana-Alonso]

¡Gracias Juan!

Forgot to mention, might also be useful to show the avg. cost per trace in an experiment there. I believe we can see avg. trace cost in the project view, but that will aggregate traces from different experiments, rendering the avg. less helpful.

[juanferrub]

Makes sense, Eduardo. Would you mind updating the original feature request description to include it too? We'll look into it

[vincentkoc]

/bounty 75

[algora-pbc[bot]]

💎 $75 bounty • Comet

Steps to solve:

1. Start working: Comment /attempt #2147 with your implementation plan
2. Submit work: Create a pull request including /claim #2147 in the PR body to claim the bounty
3. Review: Team will review PR and any clarifying questions and if successful changes will be merged
4. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

❗ Important guidelines:

• To claim a bounty, you need to provide a short demo video of your changes in your pull request
• If anything is unclear, ask for clarification before starting as this will help avoid potential rework
• Low quality AI PRs will not receive review and will be closed
• Please ask to be assigned before attempting to work on the bounty

Thank you for contributing to Comet!

Attempt	Started (UTC)	Solution	Actions
🟢 @Vikaspal8923	May 29, 2025, 04:46:25 PM	#2336	Reward

[Vikaspal8923]

@YarivHashaiComet @Eduardo-Laquintana-Alonso Hey, I am working on this bounty and updating the backend for this feature, but the changes are not reflected when I call the API from the frontend. i see the same API response as earlier. Can you help me why this is happening as I am new to oplik workflow.

[YarivHashaiComet]

hello @Vikaspal8923 , can you share more info about how you run the app? are you trying to run locally the FE & BE?

[Vikaspal8923]

@YarivHashaiComet Initially, the issue was related to running Opik locally. It has now been resolved, as I brought up my questions on Slack. I am currently working on this issue and hope to solve it soon, then raise a PR for it.

[vincentkoc]

@Vikaspal8923 let us know if you need any help in the meantime can you do the attempt command in your comment as per the bot comment above to let everyone know you are working on this.

[Vikaspal8923]

/attempt #2147

[Vikaspal8923]

@vincentkoc @YarivHashaiComet I have solved the issue locally . Need one clarification, should I use formatNumericData or just show the direct value coming in api response. I have attached a screenshot in which I use formatNumericData for the average cost and use the direct value in the Total estimated cost column. Here, the average cost becomes "0" after precision. Just let me know which one you want.

value's coming in API resp as : "total_estimated_cost": 0.000346000000, "total_estimated_cost_avg": 0.000115333333,

[algora-pbc[bot]]

🎉🎈 @Vikaspal8923 has been awarded $75 by Comet! 🎈🎊

[andrescrz]

@vincentkoc @YarivHashaiComet I have solved the issue locally . Need one clarification, should I use formatNumericData or just show the direct value coming in api response. I have attached a screenshot in which I use formatNumericData for the average cost and use the direct value in the Total estimated cost column. Here, the average cost becomes "0" after precision. Just let me know which one you want.

value's coming in API resp as : "total_estimated_cost": 0.000346000000, "total_estimated_cost_avg": 0.000115333333,

This was clarified here: https://github.com/comet-ml/opik/pull/2336#discussion_r2126771687

[andrescrz]

Hi @Eduardo-Laquintana-Alonso

This has been implemented by our contributor @Vikaspal8923:

• https://github.com/comet-ml/opik/pull/2336

It will go in our next release and deployment, planned for the next few days.

[Eduardo-Laquintana-Alonso]

Amazing @andrescrz, thanks! I do already see it in my Opik local dashboard, and the numbers look right

[andrescrz]

Amazing @andrescrz, thanks! I do already see it in my Opik local dashboard, and the numbers look right

Amazing! We've just released this change, among many others, in version 1.7.31. This version is also deployed to Opik in Comet production.