[Bug]: Discrepancy between Openrouter's deepseek-r1 and deepseek-reasoner

[ryanhoangt]

Is there an existing issue for the same bug?

• [x] I have checked the existing issues.

Describe the bug and reproduction steps

When trying to use deepseek r1 via openrouter vs. deepseek API:

• Via openrouter: the reasoning content is prepended to the actual response message, inside the <think> tag
• Via Deepseek API: the reasoning content is not presented, as it is put into reasoning_content field separately by LiteLLM

I'm not sure what is the correct way to use this reasoning_content and should we find a way to unify this? E.g. we can run evals with reasoning_content included/excluded and see how it affects performance? If it affects negatively, we can probably display it on UI only (similar to deepseek chat UI) and not send to the LLM in subsequent steps.

OpenHands Installation

Docker command in README

OpenHands Version

main

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

Response by Openrouter's deepseek-r1

Response deepseek-reasoner

No response

[enyst]

IMO we should report this issue to litellm, the bug is there, so they can unify how it sends it.

I think reasoning_content would be the right way from litellm, and then yes, as you say, we can eval how we use it.

[ryanhoangt]

Yeah, indeed there seems to be a relevant issue there: https://github.com/BerriAI/litellm/issues/8193

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for over 30 days with no activity.

[enyst]

This was just closed as completed in litellm. Looking forward to see how it works!