Feat add rag

[better629]

Features

• add rag and its example in examples/rag_pipeline.py
• apply rag to memory_storage
• update related-uts

Feature Docs

Influence

further will remove metagpt/document_store and update LTM related works.

Result

(metagpt_310) MacBook-Pro:MetaGPT xxxx$ pytest tests/metagpt/rag/*
============================================================================== test session starts ===============================================================================
collected 34 items

tests/metagpt/rag/engines/test_simple.py .....                                                                                                                             [ 14%]
tests/metagpt/rag/factories/test_base.py ..........                                                                                                                        [ 44%]
tests/metagpt/rag/factories/test_llm.py .....                                                                                                                              [ 58%]
tests/metagpt/rag/factories/test_ranker.py .....                                                                                                                           [ 73%]
tests/metagpt/rag/factories/test_retriever.py ......                                                                                                                       [ 91%]
tests/metagpt/rag/retrievers/test_bm25_retriever.py .                                                                                                                      [ 94%]
tests/metagpt/rag/retrievers/test_faiss_retriever.py .                                                                                                                     [ 97%]
tests/metagpt/rag/retrievers/test_hybrid_retriever.py .                                                                                                                    [100%]

======================================================================== 34 passed, 5 warnings in 16.35s =========================================================================




(metagpt_310) MacBook-Pro:MetaGPT xxxx$ pytest tests/metagpt/document_store/*
============================================================================== test session starts ===============================================================================
collected 8 items

tests/metagpt/document_store/test_chromadb_store.py ..                                                                                                                     [ 12%]
tests/metagpt/document_store/test_document.py .                                                                                                                            [ 25%]
tests/metagpt/document_store/test_faiss_store.py ...                                                                                                                       [ 62%]
tests/metagpt/document_store/test_lancedb_store.py .                                                                                                                       [ 75%]
tests/metagpt/document_store/test_qdrant_store.py .

========================================================================= 8 passed, 4 warnings in 17.91s =========================================================================



(metagpt_310) MacBook-Pro:MetaGPT xxxx$ pytest tests/metagpt/memory/*
============================================================================== test session starts ===============================================================================
collected 9 items

tests/metagpt/memory/test_brain_memory.py .....                                                                                                                            [ 55%]
tests/metagpt/memory/test_longterm_memory.py .                                                                                                                             [ 66%]
tests/metagpt/memory/test_memory.py .                                                                                                                                      [ 77%]
tests/metagpt/memory/test_memory_storage.py ..                                                                                                                             [100%]

========================================================================= 9 passed, 6 warnings in 10.73s =========================================================================

Other

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 74.90916% with 1243 lines in your changes are missing coverage. Please review.

Project coverage is 79.85%. Comparing base (ef53049) to head (8c218a1). Report is 1330 commits behind head on main.

Files	Patch %	Lines
metagpt/actions/rebuild_sequence_view.py	30.92%	172 Missing :warning:
metagpt/environment/mincraft_env/mincraft_env.py	27.39%	159 Missing :warning:
metagpt/rag/vector_stores/chroma/base.py	27.46%	103 Missing :warning:
...tagpt/environment/werewolf_env/werewolf_ext_env.py	60.11%	67 Missing :warning:
...tagpt/environment/mincraft_env/mincraft_ext_env.py	47.50%	63 Missing :warning:
metagpt/provider/dashscope_api.py	59.83%	49 Missing :warning:
metagpt/utils/common.py	61.06%	44 Missing :warning:
metagpt/actions/rebuild_class_view.py	19.23%	42 Missing :warning:
metagpt/schema.py	77.77%	36 Missing :warning:
metagpt/repo_parser.py	84.39%	32 Missing :warning:
... and 75 more

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #974      +/-   ##
==========================================
- Coverage   86.34%   79.85%   -6.50%     
==========================================
  Files         168      248      +80     
  Lines        9279    13765    +4486     
==========================================
+ Hits         8012    10992    +2980     
- Misses       1267     2773    +1506     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.