graphrag icon indicating copy to clipboard operation
graphrag copied to clipboard

Json validation and fix

Open s106916 opened this issue 1 year ago • 3 comments

Description

  1. This PR consolidates json string validation and loading from index, report and llm three parts into one piece.
  2. including trim extract description from llm result, replace or convert unfit chars, using json_repair for the finally repair before json loading. wrapped all functions into try_parse_json_object() as unique one function to use.
  3. There are two duplicated json_clean_up functions in both index and llm section, consolidate into one under llm model.
  4. Furthermore, wrap json_clean_up into try_parse_json_object() function without Exception for friendly retry

Related Issues

Proposed Changes

add json-repair

Checklist

  • [x] I have tested these changes locally.
  • [x] I have reviewed the code changes.
  • [x] I have updated the documentation (if necessary).
  • [ ] I have added appropriate unit tests (if applicable).

Additional Notes

[Add any additional notes or context that may be helpful for the reviewer(s).]

s106916 avatar Jul 19 '24 02:07 s106916

this is fruther consolidate version for JSON parsing: always fix all incoming json when operating _manual_json mode https://github.com/microsoft/graphrag/pull/551

s106916 avatar Jul 19 '24 02:07 s106916

Hi @AlonsoGuevara, Could you please review it? Happy to work on it and taken any suggestion.

s106916 avatar Jul 23 '24 01:07 s106916

rebased from main as requested

s106916 avatar Jul 25 '24 02:07 s106916

merge as #801

s106916 avatar Aug 03 '24 23:08 s106916