[Bug]: SecurityAnalyzer is broken and not called on tool calls

[openhands-agent]

Describe the bug and reproduction steps The SecurityAnalyzer appears to receive the event with the malicious prompt only after a completion is done. The tool call has already issued a command without proper security checks, showcasing that the SecurityAnalyzer is broken. Additionally, invariant analyzers won't launch due to permission issues.

Potential solutions Robert Brennan mentioned investigating why the tests are not identifying this issue. Meanwhile, Peter Hamilton suggested that introducing a simple analyzer using Gemini Flash might act as a safeguard. However, additional discussion is required for concrete solutions.

Logs, Errors, Screenshots, and Additional Context Attached logs from Slack revealed the issue clearly:

The invariant analyzer won't even launch. Can't fetch the container image (permission denied).

This is further supported by the discussion threads across the team validating the broken SecurityAnalyzer functionality and its outdated testing coverage.

Issue Created By: Joe Pelletier on Slack Link to Slack Thread: https://openhands-ai.slack.com/archives/C06U8UTKSAD/p1748465541550359?thread_ts=1748465541.550359&cid=C06U8UTKSAD

[jpelletier1]

We should also verify this works with the self-hosted version of OpenHands.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[jpelletier1]

Received feedback this may still be broken. Elevating to roadmap for now.

[ryanhoangt]

Hi @jpelletier1, I'll look into the issue this week.

[jpelletier1]

@xingyaoww I think we decided to prioritize #10194 over this issue, correct?

[xingyaoww]

yes!

[jpelletier1]

Closing this as duplicate of https://github.com/All-Hands-AI/OpenHands/issues/10194 - we have update the LLM Risk Analyzer in OpenHands with a new approach.