Improved rendering for Gen AI Telemetry signals in Aspire Dashboard

[sudivate]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is your feature request related to a problem? Please describe the problem.

As an AI engineer developing Generative AI applications locally using VS Code, I aim to instrument and visualize OpenTelemetry (OTEL) signals captured within my app through the Aspire dashboard. Currently, I use the Aspire dashboard to monitor all telemetry signals, particularly to visualize end-user interactions by tracking captured input and output tokens. However, the current version of the Aspire dashboard lacks a user-friendly experience when rendering large text, making analysis difficult. Improving this experience, especially in rendering telemetry signals from AI interactions, would significantly enhance usability and analysis for Gen AI Apps.

Describe the solution you'd like

A solution to this problem would be to generate a tree view that organizes all the LLM spans within the same trace or session (correlation ID) and present them in a conversational format. This structure would make it easier to follow the flow of interactions, with each span displayed sequentially alongside its associated metadata, improving clarity and allowing for better analysis of the captured telemetry signals.

Below are representational sample of user-friendly rendering from market solutions Arize

Langfuse

@leslierichardson95 @truptiparkar7

Additional context

No response

[LadyNaggaga]

To make this work here are renders we would like to be supported in the dashboard:

• [ ] Visualizing LLM Response in Trace view: Was the response is returned in the dashboard a developer needs to click on that traces and view the following:

• [ ] A card that shows: number of tokens used, Latency, Eval metrics showing up has badges the show relevancy, coherence and correctness

• [ ] Latency response can be viewed in graph that would it nice but not major priorty

• [ ] Streaming responses back from LLM and showing resources used. In the screenshot from Arize shows chat history experiences. I would like to propose is that in dashboard we extend the current table that developer gets when they click a resource they see the following in a meaningful output :

• [ ] Input ( System prompt and user prompt)
• [ ] Model used
• [ ] Generated responses

For example (not real from anywhere mocked in lists)

• [ ] An icon to easily identify your AI resources. Looking for resources in Aspire today can be overwhelming to the eye especially when you dozens for resources. Could add a :sparkles: in the structured logos for AI resources similar for what you do for databases today?

[samsp-msft]

We did improve the text rendering for long strings, especially JSON with #5018, that gives a viewer that will automatically detect json and format it appropriately.

What you seem to be asking for here is knowledge and recognition of specific span types, and then render them differently, and not just as a generic grid of properties. This is doable - and could be applied to more than just the LLM space, such as http requests or database calls.

The dashboard is (currently) a generic viewer of telemetry, regardless of the source, whereas my understanding of Arise and Langfuse is that they are specific to monitoring LLM instrumentation, and are more of an LLM specific monitoring solution, more akin to Application Insights, but just for the LLM data?

[LadyNaggaga]

@samsp-msft please look at my comment https://github.com/dotnet/aspire/issues/5902#issuecomment-2385847168 on this issue where I specified the scope of what we need for this scenario. The issue above was showing an example

[sudivate]

@LadyNaggaga @samsp-msft updated the original issue to include Prompt Tracing for post Ignite discussion.

[davidfowl]

cc @maddymontaquila @JamesNK @adamint

[dahlsailrunner]

This would be an awesome feature!! Would simplify some of our existing setup.

[davidfowl]

This is in 9.5.

@JamesNK are you going to close this issue?