2 AM, Slack Blowing Up — And It All Started With One Question
It was a Friday — the team had just finished deploying an internal system that let employees use AI chat to support their work. Everything ran fine on staging. Three hours into production, problems started: containers crashing constantly, MongoDB refusing to connect, and most critically — user conversation data was flowing straight to OpenAI’s servers with zero filtering. The CTO messaged at 2:17 AM: “Fix it now.”
That was the first time I truly understood why self-hosting an AI chat gateway matters so much for a company. Not just because of cost — but for data control, audit logs, and the ability to integrate multiple AI models without getting locked into a single provider. From that point on, before recommending AI tooling to any team, I always ask one question: whose servers is your conversation data running through?
LibreChat is the solution I chose after that. Unlike Open WebUI (which focuses on local Ollama models), LibreChat is designed to run OpenAI, Claude, Gemini, and Ollama all within a single interface. More importantly: it has an authentication system, role-based access control, and conversation management robust enough to deploy company-wide for real use — not just for demos.
What Is LibreChat and Why It’s Different
LibreChat is an open-source AI chat interface that runs entirely on your own server. The main stack includes:
- Frontend: React + TypeScript, experience close to ChatGPT
- Backend: Node.js + Express
- Database: MongoDB storing conversation history and user accounts
- File storage: Supports local, S3, or Azure Blob depending on configuration
The standout feature is the preset system: pre-configure each AI endpoint (model + system prompt + parameters), and users just select a preset and go. No more everyone on the team tweaking different system prompts and using different models — the whole team shares one baseline, making it far easier to manage.
Setting Up the Environment
Minimum Requirements
- Server: 2 vCPU, 4 GB RAM (4 vCPU / 8 GB recommended for production)
- Docker Engine 24+ and Docker Compose v2
- Domain + SSL — LibreChat does not run over HTTP in production mode
# Check version before getting started
docker --version # Docker version 24.x.x
docker compose version # Docker Compose version v2.x.xClone Repository
git clone https://github.com/danny-avila/LibreChat.git
cd LibreChatDeploying LibreChat with Docker Compose
Step 1 — Create the .env Configuration File
LibreChat uses two main config files: .env for environment variables and librechat.yaml for AI endpoints.
# Copy the sample file
cp .env.example .envOpen the .env file and update the critical values. This is the most commonly skipped step and the root cause of most issues down the line:
# MongoDB URI — keep as-is if using the bundled MongoDB container
MONGO_URI=mongodb://mongodb:27017/LibreChat
# Secret keys — MUST be randomly generated, DO NOT use default values
CREDS_KEY=f34be427ebb29de8d88c107a71546019685ed8b241d927a3ad354e35188a4a # 64 hex chars
CREDS_IV=e2341419ec3dd3d19b13a1a87fafcbfb # 32 hex chars
JWT_SECRET=16f8c0ef4a5d6b3e1f7a9c2d4e6b8f0a2c4e6b8f0a2c4e6b8f0a2c4 # 64 hex chars
JWT_REFRESH_SECRET=eaa5191f2914e082135951f8e6d6fe35ba2e21239aa7b82a3c # 64 hex chars
# Allow registration — set to false after creating all necessary accounts
ALLOW_REGISTRATION=true
# Your domain
DOMAIN_CLIENT=https://chat.yourdomain.com
DOMAIN_SERVER=https://chat.yourdomain.comUse openssl to generate keys — don’t type them manually, hand-typed keys always end up with patterns:
# Use openssl to generate random keys
openssl rand -hex 32 # for CREDS_KEY, JWT_SECRET, JWT_REFRESH_SECRET
openssl rand -hex 16 # for CREDS_IVStep 2 — Configure AI Endpoints (librechat.yaml)
This is the most important part — where you declare all your AI providers. Copy the sample file:
cp librechat.example.yaml librechat.yamlEdit librechat.yaml to match your needs:
version: 1.1.5
cache: true
endpoints:
openAI:
apiKey: "${OPENAI_API_KEY}"
models:
default: ["gpt-4o", "gpt-4o-mini", "gpt-4-turbo"]
fetch: false
titleConvo: true
titleModel: "gpt-4o-mini"
anthropic:
apiKey: "${ANTHROPIC_API_KEY}"
models:
default: ["claude-opus-4-8", "claude-sonnet-4-6", "claude-haiku-4-5"]
fetch: false
google:
apiKey: "${GOOGLE_API_KEY}"
models:
default: ["gemini-2.0-flash", "gemini-1.5-pro"]
fetch: false
# Connect to local Ollama if needed
custom:
- name: "Ollama Local"
apiKey: "ollama"
baseURL: "http://host.docker.internal:11434/v1/"
models:
default: ["llama3.1:8b", "qwen2.5:7b"]
fetch: true
titleConvo: falseAdd API keys to .env:
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...Step 3 — Start Up
# Start all services
docker compose up -d
# Monitor logs — essential for the first run
docker compose logs -f librechatThe first startup will take 2-3 minutes to pull images and initialize MongoDB. Check the status:
docker compose ps
# Expected output:
# NAME STATUS
# LibreChat-LibreChat-1 running (healthy)
# LibreChat-mongodb-1 running (healthy)Step 4 — Reverse Proxy with Nginx
LibreChat runs on port 3080. You’ll need Nginx to expose it externally with SSL:
server {
listen 443 ssl http2;
server_name chat.yourdomain.com;
ssl_certificate /etc/letsencrypt/live/chat.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/chat.yourdomain.com/privkey.pem;
location / {
proxy_pass http://localhost:3080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 300s; # Important: AI responses can take a very long time
proxy_send_timeout 300s;
}
}Pay attention to proxy_read_timeout 300s — I hit timeouts with slower models using the default 60s, causing users to see mid-response errors even when the AI was still generating.
Security Configuration for Enterprise Environments
Disable Registration After Setup
Once you’ve created all the accounts your team needs, disable open registration immediately. Many servers get hit by registration bots that drain your API keys without the owner noticing:
# In .env
ALLOW_REGISTRATION=false
# Restart to apply
docker compose restart LibreChat-LibreChat-1Rate Limiting — Protect Your API Keys from Burning Out
# Limit message count in .env
LIMIT_CONCURRENT_MESSAGES=1
LIMIT_MESSAGE_IP=100 # 100 messages/IP per 15 minutes
LIMIT_MESSAGE_USER=50 # 50 messages/user per 15 minutesEnable Email Verification (Optional)
# Configure SMTP in .env
EMAIL_SERVICE=gmail
[email protected]
EMAIL_PASSWORD=app-specific-password
[email protected]
# Require email verification on registration
EMAIL_VERIFICATION=trueTroubleshooting Common Errors
MongoDB Connection Refused Right After Startup
This error appears because the LibreChat container starts before MongoDB finishes initializing — even with depends_on and healthcheck in place, a race condition can still occur. Quick fix:
docker compose restart LibreChat-LibreChat-1Container Crash with “Invalid CREDS_KEY length” Error
The key is the wrong length. CREDS_KEY must be exactly 64 hex characters, CREDS_IV must be exactly 32 hex characters:
# Check length
echo -n "$CREDS_KEY" | wc -c # must output 64
echo -n "$CREDS_IV" | wc -c # must output 32Streaming Response Gets Cut Off Mid-Stream
Check your reverse proxy timeout first. If you’re using Cloudflare, you’ll need to bypass it or enable “Disable Response Buffering” for the chat domain — Cloudflare buffers responses by default and will cut the stream.
Backing Up Your Data
# Manual MongoDB backup
docker exec LibreChat-mongodb-1 mongodump \
--db LibreChat \
--out /data/backup/$(date +%Y%m%d) \
--gzip
# Copy backup out of container
docker cp LibreChat-mongodb-1:/data/backup ./backup
# Automated backup cron job at 3 AM
0 3 * * * docker exec LibreChat-mongodb-1 mongodump --db LibreChat --out /data/backup/$(date +\%Y\%m\%d) --gzip8 Months After That 2 AM Night
LibreChat has been running smoothly for a 25-person team ever since — no downtime, no significant incidents. Updates are also straightforward:
git pull
docker compose pull && docker compose up -dWhat I like most is that conversation history is stored on my own MongoDB — no more worrying about where sensitive data ends up. Add the preset system on top: each department has its own preset with the right system prompt and a model pre-selected for their use case — a new team member can be up and running in 10 minutes.
Initial setup takes about 1-2 hours to get everything configured correctly, but after that maintenance is essentially zero. If your team is using AI for internal work and you don’t know where that conversation data is going — this is your concrete answer.
