evals
evals copied to clipboard
Add run eval workflow
Hi, I have added a workflow to this repository based on test_eval.yaml. This workflow enables the detection of PR comments including /run {OPENAI_MODEL_NAME} and runs the new evals included in the PR on the given model. It is designed to run only for users with write/admin permissions.
The purpose of this workflow is to allow for public running of evaluations on GPT-4 and sharing the results with evaluation writers. Additionally, we could potentially expand this workflow to report the run results to the PR.
While I am unsure if this repository is currently accepting contributions other than evals, I believe this workflow could be a valuable addition. If there are any concerns, please do let me know and I will close this PR accordingly.