alena-m

[FR] Trace "hidden" prompts

3

### Willingness to contribute No. I cannot contribute this feature at this time. ### Proposal Summary When working with Mlflow Evaluation or AI agents, there are "hidden" system prompts that...

[FR] Bring your own evaluation prompt

6

### Willingness to contribute No. I cannot contribute this feature at this time. ### Proposal Summary Users should be able to override the `grading_system_prompt_template` from https://github.com/mlflow/mlflow/blob/master/mlflow/metrics/genai/prompts/v1.py. Because the current hardcoded...

At least one label specified must be in y_true

1

Hi, with the latest changes I got a new error when run `run_generation_pipeline.py` ``` Traceback (most recent call last): File "path_to_repo/AutoPrompt/run_generation_pipeline_alena.py", line 64, in best_prompt = ranker_pipeline.run_pipeline(opt.num_ranker_steps) File "path_to_repo/AutoPrompt/optimization_pipeline.py", line...