MetaGPT icon indicating copy to clipboard operation
MetaGPT copied to clipboard

Added support for the HyDE method in quey analysis for RAG plates

Open lanlanguai opened this issue 1 year ago • 3 comments
trafficstars

Features Added the HyDE method for query-analysis in the RAG module, including an example for better understanding. Fixed the issue with the static methods in TestRAGEmbeddingFactory not being callable. The previous code passed static methods as parameters for parameterized testing, but static methods are not callable objects, leading to a TypeError. This was resolved by converting static methods to regular functions and defining them outside the class. Feature Docs No additional documentation provided.

Influence As an optional process in RAG, query-analysis will rewrite queries to enhance search results.

Result All unit tests for the new features have passed. The query-analysis process in the RAG module runs smoothly, effectively rewriting and optimizing queries for better search results. Other Added a detailed description of the changes and fixes made in the submission.

lanlanguai avatar Jul 26 '24 06:07 lanlanguai

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 15.62500% with 27 lines in your changes missing coverage. Please review.

Project coverage is 55.66%. Comparing base (c0abe17) to head (2819b2e). Report is 12 commits behind head on main.

Files Patch % Lines
metagpt/rag/query_analysis/HyDE.py 0.00% 14 Missing :warning:
metagpt/rag/factories/HyDEQueryTransformFactory.py 0.00% 13 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1413       +/-   ##
===========================================
+ Coverage   30.64%   55.66%   +25.01%     
===========================================
  Files         320      323        +3     
  Lines       19426    19458       +32     
===========================================
+ Hits         5954    10831     +4877     
+ Misses      13472     8627     -4845     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Jul 26 '24 07:07 codecov-commenter

The configuration information and results from running the configurations with and without the HyDE method using metagpt/rag/benchmark/hotpotqa.py are as follows:

Model Sample_Size HyDE_Used Exact_Match F1_Score
deepseek 20 yes 0.1 0.289846
deepseek 20 no 0.1 0.265604
gpt4-o 20 yes 0.55 0.726190
gpt4-o 20 no 0.45 0.626190
gpt4-o 100 yes 0.6 0.752560
gpt4-o 100 no 0.57 0.741560

lanlanguai avatar Aug 20 '24 08:08 lanlanguai

@better629

geekan avatar Oct 20 '24 06:10 geekan