[Bug]: LLM Chat does not render messages; displays structured JSON

[teocns]

Details

Occurs with claude 3.5 w/ computer use case

🔎 Describe the Bug

🤝 Contribution

• [ ] Yes, I'd be happy to submit a pull request with these changes.
• [X] I need some guidance on how to contribute.
• [X] I'd prefer the AgentOps team to handle this update.

[teocns]

Does this still occur? @thaddavis

[thaddavis]

I would have to test to confirm.

This was when I was testing AgentOps for monitoring Computer Use Agents that use Claude.

https://www.youtube.com/watch?v=t9kz9P6cEYM

Computer Use Agents take screenshots so some of the messages that are sent to the LLM are not "text" messages" but "image" messages like the one photographed above.

I would ask the front end engineers if they have added support for rendering LLM messages besides regular text messages (like images, audio, video, etc.). If they say no then this likely is still an "issue".

Pretty much what I'm suggesting is if you send an image to an LLM then the message history in AgentOps should render the image and NOT the JSON schema of the message containing the image. Makes sense?

If no one is complaining though don't worry about this for now. Solve urgent pain first.