microsoft
Query Regarding Summary Distribution per Level
Discussed in https://github.com/microsoft/graphrag/discussions/1647
Originally posted by Teo2423 January 21, 2025 I am currently working on my undergraduate thesis in Data Science at the University of Buenos Aires. A significant portion of my thesis leverages Graph RAG, particularly focusing on the use of community reports.
To achieve this, I utilize the CLI tool to index the graph, after which I extract the reports and summaries from the create_final_community_reports.parquet file.
I have encountered an issue regarding the number of summaries per level, which seems counterintuitive, and I have been unable to locate any filters or logic in the source code that might explain this behavior.
The node distribution per level, derived from the CLI logs during the indexing process (also reflected in the indexing_logs), is as follows:
Node Distribution Level 0: 3248 nodes Level 1: 1738 nodes Level 2: 1093 nodes Level 3: 65 nodes
Summary Distribution Level 0: 27 summaries Level 1: 61 summaries Level 2: 90 summaries Level 3: 35 summaries
I find it perplexing that the level with the largest number of nodes (Level 0) has the fewest summaries. This raises questions about whether there might be a filter or threshold preventing summaries from being generated for communities below a certain level of relevance, or if I might be looking for summaries in the wrong place.
Any insights or guidance on this matter would be greatly appreciated.
Thank you for your time and support.
Best regards, Teo
Main reasoning -
- Weak Nodes – Level 0 has many nodes, but most are isolated or weakly connected.
- Filtering – Nodes without strong edges are removed before summarization.
- Summarization Needs Context – Higher levels inherit structured communities; Level 0 lacks this depth.
- Summaries Prioritize Dense Data – Strongly connected nodes form better summaries, favoring higher levels.
Fix? Loosen filtering, allow weaker nodes, or tweak ALL_CONTEXT for broader summaries.
