genkit icon indicating copy to clipboard operation
genkit copied to clipboard

[Dev UI] 'Re-run evaluation' button on evaluation details page

Open ssbushi opened this issue 10 months ago • 3 comments

Overview

Provide a button to re-run evaluation when viewing eval run details.

User goal(s)

Easy way to retry evaluations without filling the run-eval form again.

Requirements

Acceptance Criteria

  • 1 Button to rerun evaluation on eval details page.
  • 2 Autofill current eval config (dataset, actionRef, metrics and actionConfig)

ssbushi avatar Feb 05 '25 16:02 ssbushi

You can add a similar button as the "Run evaluation" button at the top right corner of the eval results page, in line with the eval name heading: Image

but with the "refresh" icon and the text "Re-run evaluation" Image

shrutip90 avatar Jun 12 '25 18:06 shrutip90

I have a few questions:

  1. Do we need to take the user back to run eval page or re-run on the eval results page?
  2. What if a file was used instead of dataset to run the eval.
  3. where can I find the batch size?

@shrutip90 @ssbushi

hritan avatar Jun 16 '25 11:06 hritan

Tagging @mjchristy for UX.

  1. Good question. Maybe have a button + menu, which default to rerun, and the drop down could let them go to the run eval page for changing the config. Something like this:

Image

  1. Disable rerun eval (or take them to the run eval page).

  2. Batch size is not part of the eval metadata yet. Ignore it for now, I will add it once ready. (Please file an issue for that)

ssbushi avatar Jun 16 '25 15:06 ssbushi