Ian Webster

Frontend was still dependent on harmCategory var. This is a breaking change as the frontend now looks at metadata instead of harmCategory.

feat(webui): ability to extract json field for display

5

Ability to extract a field from LLM JSON output For example, given this setting: An output of the form: ```json { "foo": 1, "bar": "baz" } ``` Would just display...

feat: prompt management

3

# Implement `promptfoo push` and Remote Prompts This PR introduces the ability to save and reference prompts in a central location, addressing the needs of users on larger teams. ##...

**Describe the bug** Setting `--grader` on command line causes test failure. **To Reproduce** 1. promptfooconfig.yaml: ``` description: 'minimal repo' prompts: - 'test prompt {{input}}' providers: - "openai:chat:gpt-4o" defaultTest: assert: -...

Thanks for putting together this list! This PR adds Promptfoo [open-source](https://github.com/typpo/promptfoo) and [commercial](https://promptfoo.dev/)

feat: ability to set evaluateOptions.outputVars [WIP]

7

## Summary by Sourcery Add ability to configure and filter output variables in evaluation results New Features: - Introduce outputVars option to specify which variables to include in evaluation results...

feat: gender bias plugin

3

## Summary by Sourcery Add a plugin to detect gender bias in model outputs. New Features: - Introduce a `GenderBiasGrader` to identify and score gender bias, stereotypes, and discrimination in...