Deploying Weaviate with Docker: Mastering Hybrid Search and Generative Search for AI Applications - ITFROMZERO - Share tobe shared! A hands-on guide to self-hosting Weaviate Vector Database with Docker Compose. Configure Hybrid Search (BM25 + Vector) and integrate Generative Search using Python SDK v4 for production AI.

Table of Contents

The Retrieval Bottleneck in RAG Systems

Building a chatbot or an enterprise AI assistant is straightforward. The real challenge lies in the accuracy of the responses. While Large Language Models (LLMs) excel at generating fluent text, they are prone to hallucinations when missing verifiable context.

The standard solution is Retrieval-Augmented Generation (RAG): retrieving relevant documents and feeding them into the LLM prompt. However, surfacing the right context is rarely simple. If the retrieval layer fetches incorrect or incomplete information, the downstream pipeline is guaranteed to produce flawed results.

Three Approaches to Data Retrieval for AI

System engineers typically evaluate three primary approaches:

1. Keyword Search (BM25): Familiar from systems like Elasticsearch or PostgreSQL Full-Text Search. This algorithm scores relevance based on exact term frequencies (an advanced TF-IDF variant).
2. Semantic Vector Search (Dense Vector Search): Text is converted into high-dimensional vectors (e.g., 768 or 1536 dimensions). The database calculates Cosine or Dot Product similarity to capture synonymous passages even without exact keyword overlap.
3. Hybrid Search Combined with Generative Search: Runs BM25 and Vector Search concurrently on the same query. After reranking to select the top candidates, the database directly passes the retrieved context to an LLM to generate an answer in-place.

Comparing Real-World Performance

Keyword Search (BM25)

BM25 excels at matching specific, structured strings. Need to look up an order ID like DH-2024-9981, an error code like ERR_CONNECTION_RESET, or an exact proper noun like PostgreSQL? BM25 delivers 100% precision.

Its limitations emerge when users express queries naturally. Search for “how to fix memory leak errors”, and BM25 will miss documentation titled “resolving out of memory (OOM) issues” due to zero lexical overlap.

Dense Vector Search

Vector search fills the exact gap left by BM25. By capturing deep semantic intent, vector search effortlessly links a question like “year-end bonus policies” with documentation on “Q4 compensation structure”.

However, vector embeddings tend to smooth over lexical specifics. If a user needs to find a precise software version such as v2.14.3-hotfix or an IP address like 192.168.1.105, vector search often buries the exact match on page two or three.

Hybrid Search and Generative Search in Weaviate

Combining both approaches provides the most robust architecture for production environments. Internal benchmarks consistently show that Hybrid Search improves Recall@10 by 15% to 25% compared to using either method in isolation.

Weaviate balances both strategies using an alpha parameter (ranging from 0.0 to 1.0). Querying system logs? Lower alpha toward BM25. Powering an internal knowledge-base FAQ? Shift alpha toward Vector Search.

Even better, Weaviate’s Generative Search feature turns the database into a self-contained RAG engine. With a single query call, the system retrieves relevant data and instructs the LLM to synthesize an answer, eliminating dozens of lines of tedious glue code in your backend.

Why Run Weaviate on Docker?

Self-hosting a vector database is the most effective way to manage costs when scaling past millions of records. Rather than paying per-query charges on managed platforms like Pinecone, you can run Weaviate on a modest 4 vCPU / 16GB RAM VPS for just a few dozen dollars a month.

Written in Go, Weaviate is memory-efficient and offers high-performance HNSW indexing. Packaging it with Docker guarantees a consistent environment across local development workstations and staging clusters.

Hands-On: Deploying Weaviate with Docker and Python

Step 1: Configure docker-compose.yml

Create a dedicated directory for your project:

mkdir weaviate-docker && cd weaviate-docker
touch docker-compose.yml

Open docker-compose.yml and paste the following configuration. Here, we enable both the text2vec-openai module (for automated embeddings) and the generative-openai module (for automated LLM inference):

services:
  weaviate:
    image: cr.weaviate.io/semitechnologies/weaviate:1.26.1
    command:
      - --host
      - 0.0.0.0
      - --port
      - '8080'
      - --scheme
      - http
    ports:
      - "8080:8080"
      - "50051:50051"
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
      ENABLE_MODULES: 'text2vec-openai,generative-openai'
      CLUSTER_HOSTNAME: 'node1'
    volumes:
      - weaviate_data:/var/lib/weaviate

volumes:
  weaviate_data:

Start the container in detached mode:

docker compose up -d

Port 8080 handles the REST API, while port 50051 exposes gRPC. Verify that the service is healthy with a quick curl check:

curl -s http://localhost:8080/v1/meta | grep version

Step 2: Install the Python Client v4

Weaviate’s v4 Python SDK moves all data ingestion and query execution over to gRPC. As a result, batch import throughput is 2x to 3x faster compared to the legacy v3 REST client.

pip install -U "weaviate-client>=4.7.0"

Step 3: Define the Schema and Ingest Sample Data

Create an import_data.py file. This script creates a Collection (analogous to a SQL table) and inserts sample technical documentation entries:

import os
import weaviate
from weaviate.classes.config import Configure, Property, DataType

# Configure OpenAI API key for automated vectorization and response generation
OPENAI_KEY = os.environ.get("OPENAI_API_KEY", "sk-your-openai-api-key")

client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": OPENAI_KEY}
)

try:
    # Clean up existing collection if re-running tests
    if client.collections.exists("ArticleGuide"):
        client.collections.delete("ArticleGuide")

    # Initialize collection with embedding and generative model configurations
    articles = client.collections.create(
        name="ArticleGuide",
        vectorizer_config=Configure.Vectorizer.text2vec_openai(model="text-embedding-3-small"),
        generative_config=Configure.Generative.openai(model="gpt-4o-mini"),
        properties=[
            Property(name="title", data_type=DataType.TEXT),
            Property(name="content", data_type=DataType.TEXT),
            Property(name="errorCode", data_type=DataType.TEXT),
        ]
    )

    sample_data = [
        {
            "title": "Troubleshooting OOM Issues in Kubernetes Clusters",
            "content": "When a container exceeds its memory limit, the Linux kernel issues a SIGKILL via the OOM Killer to protect host node resources.",
            "errorCode": "ERR-OOM-137"
        },
        {
            "title": "Configuring PgBouncer to Mitigate PostgreSQL Connection Spikes",
            "content": "Each idle Postgres connection consumes 2-10MB of RAM. Deploying PgBouncer pools connections and significantly reduces memory overhead.",
            "errorCode": "ERR-DB-503"
        },
        {
            "title": "AWS IAM Best Practices and Privilege Scoping",
            "content": "Enforce the Principle of Least Privilege across all Service Accounts and avoid assigning broad AdministratorAccess policies.",
            "errorCode": "ERR-SEC-401"
        }
    ]

    # Use dynamic batching for optimal ingestion throughput
    with articles.batch.dynamic() as batch:
        for item in sample_data:
            batch.add_object(properties=item)

    print("Sample data successfully imported into Weaviate!")
finally:
    client.close()

Step 4: Testing Hybrid Search

Create test_hybrid_search.py to observe how the hybrid algorithm merges exact keyword matches with contextual relevance:

import os
import weaviate

client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": os.environ.get("OPENAI_API_KEY", "sk-your-openai-api-key")}
)

try:
    articles = client.collections.get("ArticleGuide")
    
    # Query includes natural language ('out of RAM pod crash') and an exact error code ('ERR-OOM-137')
    query_text = "out of RAM pod crash ERR-OOM-137"
    
    # alpha = 0.5: balanced 50% BM25 and 50% Vector Search
    response = articles.query.hybrid(
        query=query_text,
        alpha=0.5,
        limit=2
    )

    print(f"Query results for: '{query_text}'\n")
    for obj in response.objects:
        print(f"[+] Title      : {obj.properties['title']}")
        print(f"    Error Code : {obj.properties['errorCode']}")
        print(f"    Content    : {obj.properties['content']}\n")
finally:
    client.close()

Step 5: Testing Generative Search (In-Database RAG)

In standard architectures, you must execute a search, parse the retrieved texts, interpolate them into a prompt template, and issue a separate call to OpenAI. Weaviate collapses that entire workflow into a single operation.

Two operational modes are supported:

Single Prompt: Processes each retrieved record individually (ideal for document summarization or translation tasks).
Grouped Task: Combines all top-k matches into a unified context for the LLM to synthesize (the standard pattern for RAG Q&A).

Create test_generative_search.py:

import os
import weaviate

client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": os.environ.get("OPENAI_API_KEY", "sk-your-openai-api-key")}
)

try:
    articles = client.collections.get("ArticleGuide")

    # Execute hybrid search and have gpt-4o-mini synthesize the retrieved context
    response = articles.generate.hybrid(
        query="container crashed due to out of memory",
        alpha=0.6,
        limit=2,
        grouped_task="Based on the provided documents, explain the root cause of the error and provide DevOps engineers with concise remediation steps:"
    )

    print("=== RETRIEVED DOCUMENTS ===")
    for obj in response.objects:
        print(f"* {obj.properties['title']} ({obj.properties['errorCode']})")

    print("\n=== SYNTHESIZED RESPONSE FROM LLM ===")
    print(response.generated)
finally:
    client.close()

Production Best Practices

Disable Anonymous Access: Setting AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true' is suitable only for local experimentation. In production, enforce authentication via static API keys or OAuth2/OIDC to safeguard your data.
Tuning the Alpha Weight for Your Domain: Use alpha = 0.5 as a baseline. When your data contains abundant exact identifiers, product SKUs, or medical terminology, drop alpha down to 0.3 - 0.4 to favor BM25. Conversely, for open-ended conversational chatbots, values of 0.7 - 0.8 deliver smoother semantic recall.
Running Fully Air-Gapped / On-Premise: You are not locked into OpenAI. You can add local vectorizer containers such as text2vec-transformers or text2vec-ollama directly into your docker-compose.yml. Both embeddings and inference then stay completely local, guaranteeing full data privacy.