Ian Webster

Hey @miltondp, finally figured this one out. This is happening because you only have a `latest.json` file in the outputs folder, whereas it expects a `latest.json` as well as a...

Thanks for the suggestion! I want to make sure I am understanding correctly. It sounds like essentially you could just run the `promptfoo eval` command several times. Is the catch...

Very cool, thank you for sharing! I'd like to add a community section to the promptfoo website with libraries like this one, once there is enough content.

Hi @SeanyCochrane, thanks for flagging this. You're right that there's a problem with the example in the docs - mainly that it used an inconsistent variable name. I've corrected this...

As of version 0.23.1 it is now possible to load a `vars` value directly from file using the `file://` prefix. Here's an example: ```yaml - vars: system_message: file://path/to/system_message.yaml foo: some...

@SeanyCochrane thanks, I see what you're saying here. This change 64da64d4470a6a3169a5dd3c93f4961efa2b98ec strips newlines from imported vars and I've updated the [separate test configs](https://github.com/promptfoo/promptfoo/tree/main/examples/separate-test-configs) example to include a system message import....

Would you be able to show me how you're including the system message var in your main prompt?

Conversation history support has improved, feel free to reopen/open a new issue if you're still running into problems!

This is expected behavior from nunjucks! Variables can't contain spaces and `.` is a special operator. You could probably name them `1_firstVar`, `2_secondVar`, etc. Var outputs are sorted because aside...

I'm pretty sure this is already supported, as it is a test case. It may be that the webview is not showing the substituted variable, even though the assert is...