Michael

Hi @n1n-api! Thanks for reaching out! n1n.ai sounds like a great fit for promptfoo. We'd love to have you contribute the integration. You can follow our provider guide here: https://www.promptfoo.dev/docs/contributing/#adding-a-new-provider...

Promptfoo uses mathjs to evaluate expressions in derived metrics. You can use standard math operators for operations like averaging. For details, please see the [mathjs syntax documentation](https://mathjs.org/docs/expressions/syntax.html). For example, to...

Hi @nfrancis-esure, thanks for the question! Derived metrics use [mathjs](https://mathjs.org/) syntax for mathematical expressions. The documentation has been expanded since this issue was opened and now covers this in detail:...

@DKAndreasen would be happy to accept a PR with this change and send you some swag as a thank you. Please feel free to reach out if you have questions...

Hi @strentom, the main issue (rubricPrompt being ignored with "No rubric was provided") was fixed in #1877 (v0.93.0). The follow-up issues you mentioned in your last comment are separate: 1....

@HarikrishnanK9 - good news! We now have official CrewAI support: - **Documentation**: https://www.promptfoo.dev/docs/guides/evaluate-crewai/ - **Example**: You can scaffold a complete CrewAI + Promptfoo project with: ```bash npx promptfoo@latest init --example...

Hi @Gii16! I can see what's happening - this is related to how the redteam setup UI handles testing. The UI Setup's test/preview feature tries to run a quick validation...

Hi @mariana-pineda, the original issue with `max_tokens` has been addressed, and the follow-up `reasoning_effort` error was due to using an old API version. **Root cause of both errors:** 1. Azure...

Hey @vira-s, @OCEANOUXIN, @gaborerdosi76, and @dbouquin! 👋 Thanks for reporting this issue and for your patience. I did a deep dive into this and have good news! ## TL;DR This...

## Updates Fixed model IDs to use current models: - ✅ Updated `anthropic:claude-3-5-sonnet-20241022` → `anthropic:claude-sonnet-4-5-20250929` (Claude Sonnet 4.5) - ✅ Updated `google:gemini-2.0-flash-exp` → `google:gemini-2.5-flash` (stable Gemini 2.5) - ✅ Updated...