[holmes cli] Cannot allowlist command

[grzesuav]

What happened?

Currently when I ask holmes to do something - "check the system pods" it does

AI: Let me get a more concise view of the system pods:
The AI requested 1 tool call(s).
Running tool #3 run_bash_command: kubectl get pods -n kube-system --no-headers | awk '{print $1, $2, $3}' | head -50
Refusing LLM tool call kubectl get pods -n kube-system --no-headers | awk '{print $1, $2, $3}' | head -50
  Finished #3 in 0.01s, output length: 0 characters (0 lines) - /show 3 to view contents

⚠️  Command Approval Required
Command: kubectl get pods -n kube-system --no-headers | awk '{print $1, $2, $3}' | head -50
Reason: Refusing to execute bash command. argument cmd: invalid choice: 'awk' (choose from 'wc', 'kubectl', 'aws', 'az', 'argocd', 'docker', 'helm', 'grep', 'cut', 'sort', 'uniq', 'head', 'tail', 'tr', 'base64', 'jq', 'sed')

Do you want to approve and execute this command? (y/N)

every time. I did not find a way to add awk to allowed commands`

What did you expect to happen?

Allow for session of permanent allow for awk command

How can we reproduce it (as minimally and precisely as possible)?

Not sure what determines which tools it is usings

Anything else we need to know?

Using it with holmes ask --model="anthropic/claude-opus-4-1-20250805"

[aantn]

Hi @grzesuav, Thanks for reporting. This isn't supposed to happen.

I think there are two things happening here:

1. We recently added the run_bash_command tool as an fallback where Holmes can request from the user to run additional commands not built in. We indeed do not have an allow-list for it yet, but are working on it. That said, I think the real problem is something else 👇

2. For some reason Holmes is using the run_bash_command tool (which requires user approval) instead of the builtin Kubernetes toolset (which does not require user approval as all commands in it are safe). To prevent that, can you please add the following to ~/.holmes/config:

toolsets:
  bash:
    enabled: false 

Then ask the question again with --refresh-toolsets. This will disable the bash tool, but even without it, you should be able to get good results.

I am not certain, but I suspect this might be more likley to happen with Opus, so if you want to test first with Sonnet (anthropic/claude-opus-4-20250514) that would also help us troubleshoot.

[grzesuav]

1. Checked with holmes ask --model="anthropic/claude-sonnet-4-20250514" - do not ask for any permissions
2. Thanks, will try to disable bash tooling

Having configurable allowlist of tools would be great, both with permanent allow list in config and maybe an option to allow some command for the given session

[aantn]

Having configurable allowlist of tools would be great, both with permanent allow list in config and maybe an option to allow some command for the given session

Yeah, this is the plan!

Thanks for testing with Sonnet - let me know if disabling bash tooling also works.

[aantn]

@grzesuav did disabling bash help? We're making some improvements here would love to get feedback.