Michael
@dependabot rebase
@devikcode can you please share a minimal config so I can reproduce your issue? Thanks!
Closing as stale
Hi @sudarshanbadireddi, Thanks for reporting this - the `PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION` environment variable is now working correctly on Mac (and all other platforms). You can use it like this: ```bash export PROMPTFOO_DISABLE_REDTEAM_REMOTE_GENERATION=true...
Hi @hci2, Thanks for raising these questions about the output file format! As @sklein12 mentioned, **PR #1776 was merged on October 7, 2024** which addressed your concerns: - The output...
@ghchinoy That's a great idea. I started to update the vertex docs in https://github.com/promptfoo/promptfoo/pull/3107 but fell into a rabbit hole when I realized we needed to improve compatibility for claude...
Hi @kylestratis! It's in main but not released yet. We will cut a release later today to resolve it!
Hi @kylestratis and @danielsotopino! Sorry for the delay. This was fixed. Pease pull the latest version and give it a try. Feel free to comment or re-open if you encounter...
@albertlieyingadrian how large is your eval? If you put some time on my calendar https://cal.com/michael-dangelo/30min - we can move you over to our enterprise version with private sharing
@decaffeinatedio have you tried running the example located in `examples/amazon-bedrock/promptfooconfig.nova.multimodal.yaml`? When I run that example which uses the prompt you reference, I get