Michael

Hi! Awesome work so far! I'm migrating my keystone 3.x project to the 4.x beta. These are two quick features that I think could ease migration and make code syntactically...

feat(eval): add global pass rate threshold

1

This PR addresses #1441 It introduces a configurable pass rate threshold for evals. The command now exits with PROMPTFOO_FAILED_TEST_EXIT_CODE if the pass rate falls below the specified threshold. When the...

- Upgrade @anthropic-ai/sdk from 0.25.0 to 0.26.0 - Upgrade @aws-sdk/client-bedrock-runtime from 3.623.0 to 3.631.0 - Upgrade @eslint/js from 9.8.0 to 9.9.0 - Upgrade @libsql/client from 0.8.1 to 0.9.0 - Upgrade...

Red team tests are now stored in a separate file and referenced in the main configuration, rather than being embedded directly in the existing promptfooconfig or promptfooconfig being copied into...

fix(bedrock): ensure numeric conversion for token counts

2

Affects bedrock nova

refactor(providers): load default providers from a registry

2

This refactor centralizes the logic for loading default providers, aligning it with their respective implementations. - Moved default provider configuration logic to individual provider files. - Updated provider selection logic...

feat(providers): add Slack provider

2

This PR introduces Slack integration for collecting human feedback as a provider - Added Slackbot provider for sending prompts and receiving responses via Slack. - Updated documentation and examples to...