Integration: fastRAG

fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines

Authors
Intel Labs
IntelLabs
GitHub Repo

fastRAG is a research framework, that extends Haystack, with abilities to build efficient and optimized retrieval augmented generative pipelines (with emphasis on Intel hardware), incorporating state-of-the-art LLMs and Information Retrieval modules.

Key Features

  • Optimized RAG: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
  • Optimized for Intel Hardware: Leverage Intel extensions for PyTorch (IPEX), 🤗 Optimum Intel and 🤗 Optimum-Habana for running as optimal as possible on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
  • Customizable: fastRAG is built using Haystack and HuggingFace. All of fastRAG’s components are 100% Haystack compatible.

:rocket: Components

For a brief overview of the various unique components in fastRAG refer to the Components Overview page.

LLM Backends
Intel Gaudi Accelerators Running LLMs on Gaudi 2
ONNX Runtime Running LLMs with optimized ONNX-runtime
Llama-CPP Running RAG Pipelines with LLMs on a Llama CPP backend
Optimized Components
Embedders Optimized int8 bi-encoders
Rankers Optimized/sparse cross-encoders
RAG-efficient Components
ColBERT Token-based late interaction
Fusion-in-Decoder (FiD) Generative multi-document encoder-decoder
REPLUG Improved multi-document decoder
PLAID Incredibly efficient indexing engine

:round_pushpin: Installation

Preliminary requirements:

  • Python 3.8 or higher.
  • PyTorch 2.0 or higher.

To set up the software, clone the project and run the following, preferably in a newly created virtual environment:

git clone https://github.com/IntelLabs/fastRAG.git
cd fastrag
pip install .

There are several dependencies to consider, depending on your specific usage:

# Additional engines/components
pip install .[intel]               # Intel optimized backend [Optimum-intel, IPEX]
pip install .[elastic]             # Support for ElasticSearch store
pip install .[qdrant]              # Support for Qdrant store
pip install libs/colbert           # Indexing engine for ColBERT/PLAID
pip install .[faiss-cpu]           # CPU-based Faiss library
pip install .[faiss-gpu]           # GPU-based Faiss library
pip install .[knowledge_graph]     # Libraries for working with spacy and KG

# User interface (for demos)
pip install .[ui]

# Benchmarking
pip install .[benchmark]

# Development tools
pip install .[dev]