fastRAG
fastRAG copied to clipboard
IntelLabs
Efficient Retrieval Augmentation and Generation Framework
Build and explore efficient retrieval-augmented generative models and applications
:round_pushpin: Installation • :rocket: Components • :books: Examples • :red_car: Getting Started • :pill: Demos • :pencil2: Scripts • :bar_chart: Benchmarks
fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers with a comprehensive tool-set for advancing retrieval augmented generation.
Comments, suggestions, issues and pull-requests are welcomed! :heart:
[!IMPORTANT] Now compatible with Haystack v2+. Please report any possible issues you find.
:mega: Updates
- 2024-05: fastRAG V3 is Haystack 2.0 compatible :fire:
- 2023-12: Gaudi2 and ONNX runtime support; Optimized Embedding models; Multi-modality and Chat demos; REPLUG text generation.
- 2023-06: ColBERT index modification: adding/removing documents; see IndexUpdater.
- 2023-05: RAG with LLM and dynamic prompt synthesis example.
- 2023-04: Qdrant
DocumentStoresupport.
Key Features
- Optimized RAG: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
- Optimized for Intel Hardware: Leverage Intel extensions for PyTorch (IPEX), 🤗 Optimum Intel and 🤗 Optimum-Habana for running as optimal as possible on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
- Customizable: fastRAG is built using Haystack and HuggingFace. All of fastRAG's components are 100% Haystack compatible.
:rocket: Components
For a brief overview of the various unique components in fastRAG refer to the Components Overview page.
| LLM Backends | |
| Intel Gaudi Accelerators | Running LLMs on Gaudi 2 |
| ONNX Runtime | Running LLMs with optimized ONNX-runtime |
| OpenVINO | Running quantized LLMs using OpenVINO |
| Llama-CPP | Running RAG Pipelines with LLMs on a Llama CPP backend |
| Optimized Components | |
| Embedders | Optimized int8 bi-encoders |
| Rankers | Optimized/sparse cross-encoders |
| RAG-efficient Components | |
| ColBERT | Token-based late interaction |
| Fusion-in-Decoder (FiD) | Generative multi-document encoder-decoder |
| REPLUG | Improved multi-document decoder |
| PLAID | Incredibly efficient indexing engine |
:round_pushpin: Installation
Preliminary requirements:
- Python 3.8 or higher.
- PyTorch 2.0 or higher.
To set up the software, clone the project and run the following, preferably in a newly created virtual environment:
pip install .
There are several dependencies to consider, depending on your specific usage:
# Additional engines/components
pip install .[intel] # Intel optimized backend [Optimum-intel, IPEX]
pip install .[elastic] # Support for ElasticSearch store
pip install .[qdrant] # Support for Qdrant store
pip install .[colbert] # Support for ColBERT+PLAID; requires FAISS
pip install .[faiss-cpu] # CPU-based Faiss library
pip install .[faiss-gpu] # GPU-based Faiss library
# Benchmarking
pip install .[benchmark]
# Development tools
pip install .[dev]
License
The code is licensed under the Apache 2.0 License.
Disclaimer
This is not an official Intel product.
IntelLabs