llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

Published 20 hours ago • verified publisher icon meta-llama

Improve CRUD on scoring function llm-as-judge

Open yanxi0830 opened this issue 7 months ago • 2 comments

🚀 Describe the new functionality needed

We need to refactor llm-as-judge to make it easy for user to perform CRUD operations.

  • unregister scoring functions
  • persisting judge prompts
  • migrating simpleqa judge prompts to other repo

https://github.com/meta-llama/llama-stack/pull/1405

💡 Why is this needed? What if we don't build it?

Prepare for llama-stack-evals repo

Other thoughts

No response

yanxi0830 avatar Mar 06 '25 22:03 yanxi0830