System Requirements

OpenRAG requires significant resources to run a local LLM (llama3.1:8b) and the complete infrastructure.

Hardware Configuration

MINIMUM Configuration (CPU-only Mode)

This configuration will allow the system to run but with limited performance. The LLM will take 70-90 seconds or more for every response — in CPU-only mode there is no warm-up shortcut, all queries are slow.

CPU

Minimum: 8 cores (x86_64)The LLM uses 80-100% of all cores during generation

RAM

Minimum: 16 GB

LLM (llama3.1:8b): ~5.5 GB
Services (PostgreSQL, Redis, Qdrant, MinIO): ~2 GB
Frontend Next.js: ~150 MB
OS + buffers: ~8 GB

Storage

Minimum: 50 GB SSD

Docker images: ~8 GB
Ollama model (llama3.1:8b): 4.9 GB
Embeddings: ~400 MB
Data + documents: 10+ GB

Network

Required: Stable internet connectionTo download LLM model (4.9 GB) and Docker images

RECOMMENDED Configuration (With GPU)

With an NVIDIA GPU, LLM performance is 10-50x faster. Responses take 1-3 seconds instead of 5-15s.

CPU

Recommended: 12+ cores

RAM

Recommended: 32 GBMore RAM allows loading larger models and handling more simultaneous users

GPU

Recommended: NVIDIA GPU with 12+ GB VRAM

RTX 3060 (12GB): Good for llama3.1:8b
RTX 4090 (24GB): Excellent for larger models
A100 (40/80GB): Production

Important: Requires CUDA Toolkit and nvidia-docker

Storage

Recommended: 100+ GB NVMe SSDFor better I/O performance on PostgreSQL and Qdrant

RAM Usage Breakdown (Production System)

Ollama (loaded LLM):        5.5 GB
PostgreSQL 16:              500 MB
Qdrant (928 vectors):       300 MB
Redis 7:                    100 MB
MinIO:                      200 MB
API Gateway:                200 MB
Orchestrator:               300 MB
Embedding Service:          200 MB
Frontend Admin (Streamlit): 250 MB
OS (Debian/Ubuntu):       2-4 GB
System Buffers:           4-6 GB
-----------------------------------
TOTAL:                   14-18 GB

Logiciels requis

Docker & Docker Compose

Required Software

Docker & Docker Compose (REQUIRED)

Linux (Debian/Ubuntu)
macOS
Windows + WSL2

# Update packages
sudo apt-get update

# Install Docker
sudo apt-get install -y docker.io docker-compose-plugin

# Add your user to docker group (avoids sudo every time)
sudo usermod -aG docker $USER
newgrp docker

# Verify versions
docker --version        # Required: 26.0+
docker compose version  # Required: 2.26+

IMPORTANT: After adding to docker group, you must log out and log back in for permissions to take effect.

# Install Docker Desktop for Mac
# Download: https://www.docker.com/products/docker-desktop

# Or with Homebrew
brew install --cask docker

Docker Desktop Configuration (Required):

RAM: 16 GB minimum
CPUs: 8 cores minimum
Swap: 2 GB
Disk: 60 GB

# 1. Install WSL2
wsl --install
wsl --set-default-version 2

# 2. Install Ubuntu in WSL2
wsl --install -d Ubuntu-22.04

# 3. Install Docker Desktop for Windows
# Download: https://www.docker.com/products/docker-desktop

Docker Desktop Configuration:

Enable “Use WSL 2 based engine”
RAM: 16 GB minimum
CPUs: 8 cores minimum

Git (REQUIRED)

# Linux
sudo apt-get install git

# macOS  
brew install git

# Verify
git --version

Recommended Utilities

These tools make testing and debugging easier but are not required:

# Linux (Debian/Ubuntu)
sudo apt-get install -y curl jq

# macOS
brew install curl jq

# Verify
curl --version
jq --version

Usefulness:

curl: Test REST API (HTTP requests)
jq: Parse and format JSON responses

Network Ports Used

OpenRAG uses 10 services with the following ports:

Public ports (accessible from browser)

Service	Port	URL	Description
Chat Interface	3000	http://localhost:3000	Next.js user chat + admin panel
REST API	8000	http://localhost:8000	API entry point
MinIO Console	9001	http://localhost:9001	Storage management (admin/admin123456)
Qdrant Dashboard	6333	http://localhost:6333/dashboard	Vector DB

Internal ports (between Docker containers)

Service	Port	Usage
PostgreSQL	5432	Database
Redis	6379	Cache and queues
MinIO API	9000	S3 storage
Qdrant gRPC	6334	Vector DB gRPC
Ollama	11434	LLM Server
Orchestrator	8001	Orchestration service
Embedding	8002	Embeddings service

Check if a Port is Available

# Linux/macOS
sudo lsof -i :3001

# If port is in use, find the process
sudo lsof -i :3001 | grep LISTEN

# Kill the process if necessary
sudo kill -9 <PID>

If a port is already in use, you’ll need to either stop the application using it, or modify the docker-compose.yml file to change port mappings.

GPU Support (Optional - 10-50x Performance)

NVIDIA GPU on Linux (Recommended for Production)

Install NVIDIA Container Toolkit

# Add NVIDIA repository
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configure Docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Test GPU in Docker

docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi

You should see your NVIDIA GPU information.

Modify docker-compose.yml for Ollama

# In docker-compose.yml, ollama section:
ollama:
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

With GPU: LLM responds in 1-3 seconds
Without GPU: LLM responds in 70-90 seconds or more (CPU-only, every query)

Apple Silicon (M1/M2/M3)

Ollama supports Metal acceleration on Apple Silicon. Performance is better than CPU-only but typically not as fast as NVIDIA GPUs. Configuration:

No special setup required
Docker Desktop for Mac handles acceleration
Performance: ~2-5 seconds per query

Quick Requirements Verification

Before installing OpenRAG, run these commands to verify your system:

# Docker versions (minimum required in comments)
docker --version        # Required: 26.0+
docker compose version  # Required: 2.26+

# Available RAM
free -h | grep Mem     # Required: 16GB minimum

# Disk space
df -h | grep -E '/$|/home'  # Required: 50GB minimum free

# GPU (optional)
nvidia-smi  # If you have an NVIDIA GPU

Pre-installation Checklist

Server with 16 GB+ RAM

50 GB+ SSD disk space

Docker 26.0+ installed

Docker Compose 2.26+ installed

User in docker group

Ports 8000, 3000 available

Stable internet connection (5 GB model download)

Configuration per Use Case

Simple Usage (no GPU)
Production (with GPU)
Lightweight Test Environment

Hardware:

CPU: 8 cores
RAM: 16 GB
SSD: 50 GB

Expected Performance:

Vector search: 100-200 ms
LLM response: 70-90 seconds or more (CPU-only, every query)

Ideal for: Testing, development, personal use

Hardware:

CPU: 4 cores
RAM: 8 GB
SSD: 30 GB

Required Modifications:

Use lightweight model: phi3:mini instead of llama3.1:8b
Reduce number of workers
Limit non-essential services

Performance: Limited but functional for basic tests

Next Steps

Once requirements are met, consult the Quick Start Guide to install OpenRAG in 5 minutes.

Quick Start

Install and launch OpenRAG with docker compose up

Getting Started

API Reference

Components

Experiments & Research

Tests

System Requirements

System Requirements

Hardware Configuration

MINIMUM Configuration (CPU-only Mode)

CPU

RAM

Storage

Network

RECOMMENDED Configuration (With GPU)

CPU

RAM

GPU

Storage

RAM Usage Breakdown (Production System)

Logiciels requis

Docker & Docker Compose

Required Software

Docker & Docker Compose (REQUIRED)

Git (REQUIRED)

Recommended Utilities

Network Ports Used

Public ports (accessible from browser)

Internal ports (between Docker containers)

Check if a Port is Available

GPU Support (Optional - 10-50x Performance)

NVIDIA GPU on Linux (Recommended for Production)

Apple Silicon (M1/M2/M3)

Quick Requirements Verification

Pre-installation Checklist

Configuration per Use Case

Next Steps

Quick Start

​System Requirements

​Hardware Configuration

​MINIMUM Configuration (CPU-only Mode)

CPU

RAM

Storage

Network

​RECOMMENDED Configuration (With GPU)

CPU

RAM

GPU

Storage

​RAM Usage Breakdown (Production System)

​Logiciels requis

​Docker & Docker Compose

​Required Software

​Docker & Docker Compose (REQUIRED)

​Git (REQUIRED)

​Recommended Utilities

​Network Ports Used

​Public ports (accessible from browser)

​Internal ports (between Docker containers)

​Check if a Port is Available

​GPU Support (Optional - 10-50x Performance)

​NVIDIA GPU on Linux (Recommended for Production)

​Apple Silicon (M1/M2/M3)

​Quick Requirements Verification

​Pre-installation Checklist

​Configuration per Use Case

​Next Steps

Quick Start

System Requirements

Hardware Configuration

MINIMUM Configuration (CPU-only Mode)

RECOMMENDED Configuration (With GPU)

RAM Usage Breakdown (Production System)

Logiciels requis

Docker & Docker Compose

Required Software

Docker & Docker Compose (REQUIRED)

Git (REQUIRED)

Recommended Utilities

Network Ports Used

Public ports (accessible from browser)

Internal ports (between Docker containers)

Check if a Port is Available

GPU Support (Optional - 10-50x Performance)

NVIDIA GPU on Linux (Recommended for Production)

Apple Silicon (M1/M2/M3)

Quick Requirements Verification

Pre-installation Checklist

Configuration per Use Case

Next Steps