Thomas Cleberg

Feature Request: Also Pass `logProbs` to Python Assertions

1

Currently, Python custom asserts only receive the output and the context from the provider. In order to provide full functionality, we also need to pass the `logprobs` object. https://github.com/promptfoo/promptfoo/blob/7639fbe5bb40482be7faf4f9bbe1aed2a277d2fc/src/assertions.ts#L268

I want to ensure that I can use a specific version of a specific metric (using the current prompt and generation method, for example) and be guaranteed of continuing to...

### ⚠️ Please check that this feature request hasn't been suggested before. - [X] I searched previous [Ideas in Discussions](https://github.com/axolotl-ai-cloud/axolotl/discussions/categories/ideas) didn't find any similar feature requests. - [X] I searched...