deepset-ai
Enable/refine reasoning support
In #9696, we added the ReasoningContent field to ChatMessage for storing model's reasoning.
For each ChatGenerator that explicitly supports reasoning-capable models, we should do the following:
- Verify whether init/run parameters allow reasoning settings. If not, add them.
- Refactor internal methods to ensure that reasoning is stored in the new field (not in
meta). - At runtime, handle reasoning content appropriately: some providers require removing reasoning from previous messages, while others require including it.
- Check how reasoning works with tool calls. Several providers have special requirements (e.g., using reasoning signatures).
- Check that when streaming is enabled, the reasoning content is successfully returned in the final assistant message. (For the moment, I have the impression that we cannot support streaming reasoning in a unified way).
Most child issues should wait until the 2.17.0 release (which will include the new field).
The list of child issues might not be comprehensive. If I forgot some relevant ChatGenerator, feel free to add it.
Hey, just want to put my two cents in. I'd consider that reasoning content is somewhat different to other content types because of the following reasons:
- it can only be a string type
- it always happen before the actual output (if you want to separate reasoning time and the actual response time)
- it is generally not considered helpful when carrying on the conversation (that is, the reasoning part should most likely be dropped in follow-up rounds)
- By choosing which model what generation arguments to use, we should know in advance if this has reasoning or not.
- In most of the APIs (I am talking about Gemini and DeepSeek but not Claude - Claude is kind of a special case), it is not included as a part of the content types but rather supplementary.
IMO it should not be included as a part of _content but rather an independent field. There is a special case for Claude models though: Anthropic introduces a signature token attached to the thinking process (or redacted) if it is carried on during tool calls, so therefore it is more important comparing to the other reasoning models.
I hope it's not too late, but this is what I thought and what I did in my approach months ago and it has been working perfectly fine.
@anakin87 as a follow up for issues that haven't been closed yet we should probably also try and adopt using the new reasoning field of StreamingChunk as well. Opened issue for that here https://github.com/deepset-ai/haystack/issues/9785
