Qwen3 thinks correctly but doesn't actually tool call

[pcuci]

Wanted to run qwen3:32b-q4_K_M (fits on a 24 GiB VRAM Radeon 7900 XTX card, serving via Ollama). While it's reasoning correctly about what it needs to do, it doesn't actually call the tool. Even though it appears to have been trained specifically for tool use: https://github.com/QwenLM/Qwen3?tab=readme-ov-file#tool-use

Wondering if the < think ></ think > tags get in the way?

PS: If you're on an older Ollama, might need to upgrade to avoid blob errors: https://github.com/ollama/ollama/issues/10454#issuecomment-2837175276

👍 if useful

[enyst]

Could you try to set this to true, and see how it works with it? https://github.com/All-Hands-AI/OpenHands/blob/5fa01ed278fea17acf38f0b6092b13cd3e466ad5/config.template.toml#L201

[pcuci]

Same behaviour

Is there a way to check that my configuration was indeed picked up for sure by the OpenHands runtime?

root@4e4639c4d127:/app# cat config.toml | grep native_tool_calling
native_tool_calling = true
root@4e4639c4d127:/app# echo $CONFIG_FILE
/app/config.toml

[pcuci]

@enyst - can we override FUNCTION_CALLING_SUPPORTED_MODELS at runtime?

perhaps qwen3 needs to be added here?

[enyst]

@pcuci We should be able to override from env almost all options in config.toml (all the primitives / simple enough ones)

This one is in llm section, so please use -e LLM_NATIVE_TOOL_CALLING=true in the docker command, or export it in your env. You're running with docker run ?

[enyst]

@enyst - can we override FUNCTION_CALLING_SUPPORTED_MODELS at runtime?

perhaps qwen3 needs to be added here?

Yes, and the env var does that too: https://github.com/All-Hands-AI/OpenHands/blob/760a14482e1c09880bb1efda305c3425744aeea4/openhands/llm/llm.py#L488

Hmm, it also depends on whether litellm supports function calling for Qwen3 yet

[enyst]

@pcuci We just merged https://github.com/All-Hands-AI/OpenHands/issues/8424 which also achieves this, because we changed the behavior so that what the user specifies (true or false) will always be followed.

[pcuci]

ollama/qwen3:30b-a3b-q4_K_M Works on OpenHands 0.38.0, with LLM_NATIVE_TOOL_CALLING=true

Thank you, @enyst!

[0xNOY]

Hi,

I'm encountering an issue related to tool calling with an Ollama model.

I'm currently using the ollama/qwen3:32b-fp16 model. When I run OpenHands without the LLM_NATIVE_TOOL_CALLING=true flag, the LLM responds, but it does not perform any tool calls, which is the original problem I was trying to solve.

Following suggestions to enable native tool calling, I added the -e LLM_NATIVE_TOOL_CALLING=true flag to my docker run command. My current startup command is:

docker run -it --rm --pull=always \
        -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.38-nikolaik \
        -e LOG_ALL_EVENTS=true \
        -v /var/run/docker.sock:/var/run/docker.sock \
        -v ~/.openhands-state:/.openhands-state \
        -p 3000:3000 \
        --add-host host.docker.internal:host-gateway \
        --name openhands-app \
        --network host \
        -e LLM_NATIVE_TOOL_CALLING=true \
        docker.all-hands.dev/all-hands-ai/openhands:0.38

After adding the LLM_NATIVE_TOOL_CALLING=true flag, I now encounter a new error during the agent's operation, and the agent fails to proceed:

10:48:35 - openhands:ERROR: agent_controller.py:307 - [Agent Controller 2f1285646bb9494988d293975914095f] Error while running the agent (session ID: 2f1285646bb9494988d293975914095f): litellm.APIConnectionError: 'str' object has no attribute 'get'

Details

The traceback indicates an AttributeError: 'str' object has no attribute 'get' within the litellm library, specifically in the convert_content_list_to_str function at the line text_content = c.get("text").

This suggests that litellm, when preparing the request for the Ollama provider (in this case, ollama/qwen3:32b-fp16), expects a dictionary-like object for a part of the message content but receives a plain string instead. This error only appears when LLM_NATIVE_TOOL_CALLING=true is set.

It seems that enabling LLM_NATIVE_TOOL_CALLING might be altering the message structure passed to litellm in a way that is incompatible with its Ollama message transformation logic, specifically for how message content (perhaps system messages or parts of user/assistant messages) is structured.

Could this new error be a side effect of how LLM_NATIVE_TOOL_CALLING=true interacts with litellm's handling of messages for Ollama models like qwen3:32b-fp16? Any insights or guidance on how to resolve this to get tool calling to work with this setup would be greatly appreciated.

Please find the full traceback below for more details:

10:48:35 - openhands:ERROR: agent_controller.py:307 - [Agent Controller 2f1285646bb9494988d293975914095f] Error while running the agent (session ID: 2f1285646bb9494988d293975914095f): litellm.APIConnectionError: 'str' object has no attribute 'get'
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2904, in completion
    response = base_llm_http_handler.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 270, in completion
    data = provider_config.transform_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/ollama/completion/transformation.py", line 322, in transform_request
    modified_prompt = ollama_pt(model=model, messages=messages)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 217, in ollama_pt
    system_content_str, msg_i = _handle_ollama_system_message(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 175, in _handle_ollama_system_message
    msg_content = convert_content_list_to_str(messages[msg_i])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/common_utils.py", line 116, in convert_content_list_to_str
    text_content = c.get("text")
                   ^^^^^
AttributeError: 'str' object has no attribute 'get'
. Traceback: Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2904, in completion
    response = base_llm_http_handler.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 270, in completion
    data = provider_config.transform_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/ollama/completion/transformation.py", line 322, in transform_request
    modified_prompt = ollama_pt(model=model, messages=messages)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 217, in ollama_pt
    system_content_str, msg_i = _handle_ollama_system_message(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 175, in _handle_ollama_system_message
    msg_content = convert_content_list_to_str(messages[msg_i])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/common_utils.py", line 116, in convert_content_list_to_str
    text_content = c.get("text")
                   ^^^^^
AttributeError: 'str' object has no attribute 'get'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/openhands/controller/agent_controller.py", line 305, in _step_with_exception_handling
    await self._step()
  File "/app/openhands/controller/agent_controller.py", line 827, in _step
    action = self.agent.step(self.state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/agenthub/codeact_agent/codeact_agent.py", line 211, in step
    response = self.llm.completion(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 338, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 477, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 378, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 400, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 480, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/llm/llm.py", line 280, in wrapper
    resp: ModelResponse = self._completion_unwrapped(*args, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1255, in wrapper
    raise e
  File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1133, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 3216, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2224, in exception_type
    raise e
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2200, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: 'str' object has no attribute 'get'
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2904, in completion
    response = base_llm_http_handler.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 270, in completion
    data = provider_config.transform_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/ollama/completion/transformation.py", line 322, in transform_request
    modified_prompt = ollama_pt(model=model, messages=messages)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 217, in ollama_pt
    system_content_str, msg_i = _handle_ollama_system_message(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 175, in _handle_ollama_system_message
    msg_content = convert_content_list_to_str(messages[msg_i])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/prompt_templates/common_utils.py", line 116, in convert_content_list_to_str
    text_content = c.get("text")
                   ^^^^^
AttributeError: 'str' object has no attribute 'get'

Thanks for your help.

[enyst]

When I run OpenHands without the LLM_NATIVE_TOOL_CALLING=true flag, the LLM responds, but it does not perform any tool calls, which is the original problem I was trying to solve.

That's an odd error, it looks like litellm doesn't support it. Maybe you could also report this to litellm project?

If LLM_NATIVE_TOOL_CALLING is not true, does the model "execute" actions or does it only talk?

[0xNOY]

That's an odd error, it looks like litellm doesn't support it. Maybe you could also report this to litellm project?

Yes, I agree this AttributeError with LLM_NATIVE_TOOL_CALLING=true seems related to litellm. I plan to report this to the litellm project as well.

If LLM_NATIVE_TOOL_CALLING is not true, does the model "execute" actions or does it only talk?

When LLM_NATIVE_TOOL_CALLING is not true, the model only talks and does not execute actions. The behavior is exactly as shown in the initial image of this Issue: the LLM reasons about using a tool but doesn't actually call it.

The AttributeError only appears when LLM_NATIVE_TOOL_CALLING=true is set.

Thanks,

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[BradKML]

Is this still an issue for Qwen3, when both OpenRouter and Ollama support tool calling (and thinking/reasoning) nowadays? Also what would be the ideal -e FLAGS assuming that everything from temperature, Top P, Top K, context size, and tweaking thinking + tool calling is needed?

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[BradKML]

After playing around with RooCode + RooFlow once again, this problem is likely an issue with prompting/config that runs across IDEs. Even if they use tools correctly, they are not "self-motivated" to finish a checklist of tasks, which is weird.

[BradKML]

@0xNOY could you check this out just in case? LiteLLM should be supporting this normally, and Ollama also has support. So something else has to be in play, maybe something lost in transport? https://docs.litellm.ai/docs/providers/ollama#example-usage---tool-calling

[BradKML]

GPT-OSS is confirmed to work, and also Qwen3-Coder has some whacky formats similar to GPT-OSS, so maybe there is a way around this? But Qwen3 (instruct) has less restrictions https://github.com/All-Hands-AI/OpenHands/issues/10112

[enyst]

Brad, could you please tell, what is the remaining issue here? I mean, these models don’t have tool calling enabled by default, but the user can enable it with the LLM_NATIVE_TOOL_CALLING=true parameter.

If they seem not “self-motivated” to use tools, that sounds like a matter of prompting, what if you tried to add a microagent that yells at the LLM a bit? 😅 Well, I mean, tell it promptly like “You MUST use the tools available to you to fulfill the task.” Perhaps also “Use the task tool to keep track of your tasks, and finish it. You MUST finish the user task.”

[BradKML]

Has anyone tried Qwen3-Next on this issue as well?

[github-actions[bot]]

This issue is stale because it has been open for 40 days with no activity. Remove the stale label or leave a comment, otherwise it will be closed in 10 days.

[github-actions[bot]]

This issue was automatically closed due to 50 days of inactivity. We do this to help keep the issues somewhat manageable and focus on active issues.

[BradKML]

@enyst I have noticed that this issue is very common across different tools, and that even trying to motivate them fails. I suspect that it has something to do with the model training itself that is causing this issue.