MetaGPT
MetaGPT copied to clipboard
Added support for the HyDE method in quey analysis for RAG plates
Features Added the HyDE method for query-analysis in the RAG module, including an example for better understanding. Fixed the issue with the static methods in TestRAGEmbeddingFactory not being callable. The previous code passed static methods as parameters for parameterized testing, but static methods are not callable objects, leading to a TypeError. This was resolved by converting static methods to regular functions and defining them outside the class. Feature Docs No additional documentation provided.
Influence As an optional process in RAG, query-analysis will rewrite queries to enhance search results.
Result All unit tests for the new features have passed. The query-analysis process in the RAG module runs smoothly, effectively rewriting and optimizing queries for better search results. Other Added a detailed description of the changes and fixes made in the submission.
:warning: Please install the to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 15.62500% with 27 lines in your changes missing coverage. Please review.
Project coverage is 55.66%. Comparing base (
c0abe17) to head (2819b2e). Report is 12 commits behind head on main.
| Files | Patch % | Lines |
|---|---|---|
| metagpt/rag/query_analysis/HyDE.py | 0.00% | 14 Missing :warning: |
| metagpt/rag/factories/HyDEQueryTransformFactory.py | 0.00% | 13 Missing :warning: |
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@ Coverage Diff @@
## main #1413 +/- ##
===========================================
+ Coverage 30.64% 55.66% +25.01%
===========================================
Files 320 323 +3
Lines 19426 19458 +32
===========================================
+ Hits 5954 10831 +4877
+ Misses 13472 8627 -4845
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
The configuration information and results from running the configurations with and without the HyDE method using metagpt/rag/benchmark/hotpotqa.py are as follows:
| Model | Sample_Size | HyDE_Used | Exact_Match | F1_Score |
|---|---|---|---|---|
| deepseek | 20 | yes | 0.1 | 0.289846 |
| deepseek | 20 | no | 0.1 | 0.265604 |
| gpt4-o | 20 | yes | 0.55 | 0.726190 |
| gpt4-o | 20 | no | 0.45 | 0.626190 |
| gpt4-o | 100 | yes | 0.6 | 0.752560 |
| gpt4-o | 100 | no | 0.57 | 0.741560 |
@better629