graphrag
graphrag copied to clipboard
Json validation and fix
Description
- This PR consolidates json string validation and loading from index, report and llm three parts into one piece.
- including trim extract description from llm result, replace or convert unfit chars, using json_repair for the finally repair before json loading. wrapped all functions into try_parse_json_object() as unique one function to use.
- There are two duplicated json_clean_up functions in both index and llm section, consolidate into one under llm model.
- Furthermore, wrap json_clean_up into try_parse_json_object() function without Exception for friendly retry
Related Issues
Proposed Changes
add json-repair
Checklist
- [x] I have tested these changes locally.
- [x] I have reviewed the code changes.
- [x] I have updated the documentation (if necessary).
- [ ] I have added appropriate unit tests (if applicable).
Additional Notes
[Add any additional notes or context that may be helpful for the reviewer(s).]
this is fruther consolidate version for JSON parsing: always fix all incoming json when operating _manual_json mode https://github.com/microsoft/graphrag/pull/551
Hi @AlonsoGuevara, Could you please review it? Happy to work on it and taken any suggestion.
rebased from main as requested
merge as #801