[Feature Request] Improve Evaluation Charts

[hayescode]

Is your feature request related to a problem? Please describe. My evaluation flow has inputs like "source system" and "difficulty" that i'd like charts for to visually see progress across evaluations. The only fields that are available in the Custom Charts however are evaluation flow outputs, not even my custom metrics.

Describe the solution you'd like Any field in the evaluation flow results should be available for custom charts, including custom metrics (aggregations).

Additional context

[vinuthakaranth]

Hi @hayescode , thanks for your valuable feedback in enhancing our product.

We are trying to understand what 'including custom metrics (aggregations)' means here. How are you envisioning having this in chart? Any further details on this would definitely help in putting this request to implementation.

Could you give us an example of x-Axis and y-axis combinations you would like to see in your custom chart and how that would help you in viewing progress?

[hayescode]

Hi @vinuthakaranth - I have dimensions like difficulty (easy/medium/hard) I'd like to so the accuracy across for example. Another example would be RAG source, so I can see source A has high accuracy but not source B, soay e I should look into Source B instead of anything with the prompting/LLM.

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!