oso icon indicating copy to clipboard operation
oso copied to clipboard
verified publisher icon opensource-observer

Enable evals to run automatically for every deployment

Open ravenac95 opened this issue 8 months ago • 1 comments

What is it?

Evals should be running for every deployment much like we run tests. This should only happen on pushes to production.

Some questions:

  • Should this just run in a separate container (like a job?)
  • Should we run this as some endpoint?

ravenac95 avatar May 08 '25 16:05 ravenac95

OSO-712 Enable evals to run automatically for every deployment

linear[bot] avatar May 08 '25 16:05 linear[bot]

Going to deprioritize this for now. We have !run_eval in Discord

ryscheng avatar May 28 '25 20:05 ryscheng

I think we need to decide how often we want to run these.

ryscheng avatar Sep 17 '25 02:09 ryscheng