AgentOps-AI
Crew support missing data
I am seeing inconsistent results compared to what the CLI informs for my crew. This one I built from scratch using agentstack and has the original Poetry implementation.
Since I had auto update disabled, i updated to 0.2.3 manually thinking it would fix it, and it did but just for one execution. I am not currently able to visualize anything but the LLM calls, even though they are properly tracked in the CLI.
I have been noticing different behaviors during the last hour, ranging from not displaying info at all un the Web UI, to not detecting tools, to working just fine.
I tested another crew from a different environment and project folder, and also worked in that weird way.
First crew: This one does not show the Tools (Serper) and error the CLI informs (and i can confirm they did run just as the CLI says). 🖇 AgentOps: Session Stats - Duration: 4m 11.7s | Cost: $0.044562 | LLMs: 9 | Tools: 6 | Actions: 0 | Errors: 2 🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=a4adfab2-bfa7-44e4-9498-da9255d5bd5c PastedGraphic-2.png
Second crew: this one did not detect tools (custom tools) in the CLI or the Web UI. Just detected LLM calls.
🖇 AgentOps: Session Stats - Duration: 2m 2.2s | Cost: $0.040895 | LLMs: 8 | Tools: 0 | Actions: 0 | Errors: 0 🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=0f3ee714-4b24-4e4e-aad2-14771edf2d2f
PastedGraphic-1.png
——
Update: after I drafted this email, one of the crews was monitored correctly but just once. However, I ran it again just to make sure and this happened: 🖇 AgentOps: Session Stats - Duration: 1m 32.5s | Cost: $0.046415 | LLMs: 6 | Tools: 3 | Actions: 0 | Errors: 0 🖇 AgentOps: Session Replay: https://app.agentops.ai/drilldown?session_id=a7aa4cd3-b3c6-4b50-9f9e-3a75578c881f PastedGraphic-3.png
Should I open an issue in the AgentOps repo? I also made sure to use different API keys from AgentOps for each Crew and also different tags. I also ran them sepparetely instead of concurrently in case that was affecting the monitoring.
Chances are this will get indirectly fixed in v0.4; giving this high priority in the upcoming testing suite. Thanks for reporting 🙏
