nano-graphrag icon indicating copy to clipboard operation
nano-graphrag copied to clipboard

robust json parsing & entity extraction progress log

Open rangehow opened this issue 1 year ago • 1 comments

  1. The JSON generated by large models may have issues such as being incomplete, redundant, or having formatting errors. The original solution was too brief and provided few prompts, so we made some improvements to address these.
  2. Entity extraction is the most time-consuming part of this project, but there was no explicit progress indicator. We added a percentage indicator to let users know the current progress more clearly.

rangehow avatar Sep 20 '24 06:09 rangehow

Codecov Report

Attention: Patch coverage is 90.74074% with 5 lines in your changes missing coverage. Please review.

Project coverage is 94.25%. Comparing base (1a85851) to head (ba3b2be). Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
nano_graphrag/_utils.py 90.56% 5 Missing :warning:
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #55   +/-   ##
=======================================
  Coverage   94.25%   94.25%           
=======================================
  Files          12       12           
  Lines        1288     1288           
=======================================
  Hits         1214     1214           
  Misses         74       74           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Sep 20 '24 06:09 codecov[bot]

I didn't see this PR until now, yesterday I ended up adding tqdm to every task so that all the progress bars were available by default. Happy to help with this if needed :+1:

NazimHAli avatar Oct 15 '24 23:10 NazimHAli

I didn't see this PR until now, yesterday I ended up adding tqdm to every task so that all the progress bars were available by default. Happy to help with this if needed.

Hi @NazimHAli. You can bring up a new PR, and I will reorganize the submitted code by removing the percentage related parts. But it's better to do it quickly, because tonight this PR will be merged.

rangehow avatar Oct 17 '24 06:10 rangehow

@gusye1234 cc

rangehow avatar Oct 18 '24 08:10 rangehow

I didn't see this PR until now, yesterday I ended up adding tqdm to every task so that all the progress bars were available by default. Happy to help with this if needed.

Hi @NazimHAli. You can bring up a new PR, and I will reorganize the submitted code by removing the percentage related parts. But it's better to do it quickly, because tonight this PR will be merged.

I won't have time to do it before this gets merged. You can continue with this change and I can do a follow-up later.

NazimHAli avatar Oct 18 '24 12:10 NazimHAli