Michael

@sert121 this is a great idea! We already own [promptfoo on PyPI](https://pypi.org/project/promptfoo/) and have thought about generating mypy or even pydantic types from our schema using tools like the [datamodel...

Fixed the issues raised in code review: **1. Export logs command now respects PROMPTFOO_LOG_DIR** (chatgpt-codex-connector feedback) - Updated `src/commands/export.ts` to use the same logic as `setupLogDirectory()` - `promptfoo export logs`...

Hi @oanastrut, try running: ```sh npx promptfoo generate dataset -c promptfooconfig.yaml -o tests.yaml --provider file://provider.py ``` We use the grading provider (not the providers listed in `promptfooconfig.yaml`) to generate datasets....

@oanastrut feel free to grab a time on my calendar and I'll look at this with you: [cal.com/michael-dangelo/15min](https://cal.com/michael-dangelo/15min).

Thanks for reporting this and for the follow-up from @bennibbelink. The `--provider` flag specifies the LLM used to *generate* test cases - it's separate from the providers in your config...

@HasiniRangana I am sorry we haven't gotten to this yet. Given the conflicts and how it's not editable by us, I am going to close it, but please feel free...

Hi @LucasRavCoreoz This was fixed! Please pull the latest version and give it a try. Thanks!

Hi @funmise, Thanks for your question about using Azure OpenAI for model-graded assertions. You can specify Azure as the grader provider in your assertions: ```yaml tests: - assert: - type:...

Hi @funmise, Thanks for your question about uploading files for testing! As @nt9142 pointed out in the comments, you can load file contents as variables and reference them in your...