agenta icon indicating copy to clipboard operation
agenta copied to clipboard

[AGE-1463] [bug] LLM-as-a-judge require correct_answer to run even if not in the inputs

Open mmabrouk opened this issue 8 months ago • 0 comments

Currently you cannot run an LLM-as-a-judge unless your test sets happen to have a correct_answer column. Yet the LLM-as-a-judge does not need it.

Correct behavior: It should be possible to run an llm as a judge without needing a correct_answer

From SyncLinear.com | AGE-1463

mmabrouk avatar Apr 18 '25 11:04 mmabrouk