genkit
genkit copied to clipboard
Checks testapp for evaluators
Just adding a test application which configures the Checks evaluators.
run genkit eval:run input-file.json --evaluators=checks/dangerous_content,checks/pii_soliciting_reciting,checks/harassment,checks/sexually_explicit,checks/hate_speech,checks/medical_info,checks/violence_and_gore,checks/obscenity_and_profanity
Where input-file.json is a file of this format:
[
{
"testCaseId": "test_case_id_1",
"input": "The input to your model.",
"output": "Example model output which. This is what will be evaluated."
}
]
Checklist (if applicable):
- [ ] Tested (manually, unit tested, etc.)
- [ ] Docs updated
I am consolidating all evaluator testapps into one... https://github.com/firebase/genkit/pull/1294
Since Checks is not very different from existing evals, can we just reuse this testapp instead? You can add a separate Checks evaluator section to the README file.