How to Install Open WebUI with Docker: Building an Internal Enterprise AI Gateway - ITFROMZERO - Share tobe shared! A step-by-step guide to deploying Open WebUI with Docker as an internal corporate AI Gateway—cutting subscription costs, preventing source code leaks, and managing access to both local and cloud models.

Table of Contents

Context & Practical Problem

About 6 months ago, our team had 35 members ranging from developers to marketers. Every month, the company’s credit card statements racked up nearly $700 for individual ChatGPT Plus subscriptions. Cost was one issue, but the bigger concern was that teammates routinely pasted database schemas and proprietary source code into public cloud services.

The best way forward back then: set up an internal AI Gateway directly on the company’s private server. Pairing Open WebUI with Docker completely solved three major problems:

Data control: All chat histories, uploaded files, and embeddings remain strictly within the internal database. Not a single prompt leaks out without being reviewed.
Flexible model switching: Handling sensitive internal documents? Run Llama 3.1 8B or Qwen 2.5 via a local Ollama instance. Need to write complex code or analyze lengthy reports? Switch to GPT-4o or Claude 3.5 Sonnet through a shared API pool with a single click.
Cost reduction and access control: Admins can easily assign department quotas, configure Admin/User roles, and restrict expensive models instead of handing out API keys indiscriminately.

Installation with Docker Compose

After struggling with bare-metal Python venv installations that ran into CUDA conflicts and C++ library headaches, I learned the hard way: always go with Docker Compose. It keeps the host clean, and upgrading or migrating servers takes less than 5 minutes.

First, prepare the project directory on your Ubuntu/Debian server:

mkdir -p /opt/ai-gateway/data && cd /opt/ai-gateway

Create a docker-compose.yml file configured for production:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: always
    ports:
      - "127.0.0.1:3000:8080"
    environment:
      - WEBUI_SECRET_KEY=generate_a_very_long_random_string_here_32_chars
      - DATA_DIR=/app/backend/data
      - ENABLE_SIGNUP=true
      - DEFAULT_MODELS=llama3.1:8b
      - OPENAI_API_BASE_URLS=http://host.docker.internal:11434/v1;https://api.openai.com/v1
      - OPENAI_API_KEYS=ollama;sk-proj-your-openai-api-key-here
    volumes:
      - ./data:/app/backend/data
    extra_hosts:
      - "host.docker.internal:host-gateway"
    logging:
      driver: "json-file"
      options:
        max-size: "50m"
        max-file: "5"

Launch the container in the background:

docker compose up -d

Wait about a minute for the container to pull the image and run database migrations. Then, check the startup status with docker compose logs -f.

Detailed Post-Installation Configuration

The first person to sign up through the web interface automatically receives Root Admin privileges. Once registered, head over to configure these essential settings immediately.

1. Multi-Model Connections (Local GPU & Cloud API)

Go to Admin Panel > Settings > Connections to verify your connections:

Local Ollama: By default, the configuration uses http://host.docker.internal:11434 to reach the Ollama service running directly on the host machine. If you run a dedicated GPU server on your local network (such as a machine equipped with an RTX 4090/3090), update this to its private IP, like http://192.168.1.100:11434.
OpenAI or LiteLLM: Add endpoint URLs and keys directly to the list. If your team uses a proxy gateway such as LiteLLM or vLLM, Open WebUI is 100% compatible with the standard /v1 format.

In practice, consolidating these connections into a single gateway eliminates the need for developers and testers to manually copy-paste API keys across different applications.

2. Restricting Signups and Allocating Models

Avoid leaving signups open to everyone:

Immediately toggle off Enable New Sign Ups under Settings > General (or set ENABLE_SIGNUP=false in docker-compose and restart).
Create user accounts manually under the Users tab.
Assign default model access: Regular developers are granted access only to local Ollama models to save budget, while Tech Leads or Product Managers are unlocked for GPT-4o or Claude 3.5 Sonnet.

3. Blocking Direct Ports and Securing with Nginx Reverse Proxy

Never expose port 3000 directly to the internet. Configure Nginx to provide SSL termination and point to an internal domain:

server {
    listen 80;
    server_name ai.company.internal;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name ai.company.internal;

    ssl_certificate /etc/ssl/certs/company_ai.crt;
    ssl_certificate_key /etc/ssl/certs/company_ai.key;

    client_max_body_size 100M;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Required to prevent SSE and streaming token bottlenecks
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 600s;
        proxy_buffering off;
        proxy_cache off;
    }
}

The directive proxy_buffering off; is critical. Without it, Nginx buffers chunks before returning them, preventing smooth, real-time token-by-token streaming.

Testing & Production Operations

Before sharing the link company-wide, thoroughly test these three load scenarios:

1. Resource Usage Benchmarking

Monitor the container with standard Docker commands:

# View real-time logs
docker logs -f --tail=100 open-webui

# Monitor RAM and CPU usage
docker stats open-webui

At idle, Open WebUI consumes around 350MB to 600MB of RAM. When about 10 users concurrently upload PDF documents for Retrieval-Augmented Generation (RAG) queries, RAM usage can spike to 1.8GB – 2.2GB. Allocate at least 4GB of RAM on your server dedicated to this service.

2. Verifying Streaming and File Uploads

Test-chat with Llama 3.1: Measure latency from clicking send to the first token appearing. Ensure the container connects seamlessly to Ollama over the bridge network.
Switch to GPT-4o: Verify smooth streaming responses without interruptions or 60-second timeouts.
Drag and drop a ~15-page financial report PDF: Ask specific factual questions about the document to confirm the backend handles text chunking and embeddings accurately.

3. Periodic Data Backups

All chat messages, user accounts, and uploaded documents are stored in /opt/ai-gateway/data. Simply configure a nightly cron job at 2:00 AM to rsync this directory to a NAS or S3 bucket for dependable, long-term operation.