feat: add thinking mode support to fact retrieval messages
Description
This PR fixes the JSON parsing error in new_retrieved_facts reported in issue #3564.
Root Cause:
The issue occurs with models like Qwen3-8B (as reported in #3564) that in reasoning/thinking mode return non-JSON tokens before the actual JSON response. These additional tokens cause the json.loads() call to fail with "Expecting value: line 1 column 1 (char 0)".
Solution:
This PR adds a configurable thinking_mode parameter that handles reasoning mode outputs by instructing models to include thinking in their structured JSON response, preventing JSON parsing failures.
Technical Changes:
- Added
enable_thinking_modeconfiguration parameter - Modified
get_fact_retrieval_messages()to conditionally prepend thinking instruction - Updated unit tests to verify thinking mode functionality
Fixes #3564
@Vedant817 thank you for the effort but please re write the pr description the issue number seems to be misleading and keep it to the point, this will be easier for us as maintainers. Thanks!
@Vedant817 thank you for the effort but please re write the pr description the issue number seems to be misleading and keep it to the point, this will be easier for us as maintainers. Thanks!
Yes, I'll make the changes. I found I made some mistakes in the description.
@Vedant817 thank you for the effort but please re write the pr description the issue number seems to be misleading and keep it to the point, this will be easier for us as maintainers. Thanks!
Hey @parshvadaftari I have resolved the asked changes and updated the PR description, please review it.