Quick Start: Running RedisVL in 5 Minutes
Every LLM call costs 2–3 seconds and a certain number of tokens. Multiply that by thousands of requests per day, and the numbers add up fast. RedisVL is a Python library that turns Redis into a vector store and semantic cache — and the key insight is: two questions that are worded differently but mean the same thing will return cached results instead of hitting the LLM again.
Install and run Redis Stack (requires the RediSearch module):
pip install redisvl
# Start Redis Stack via Docker
docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latestTest semantic cache right away:
from redisvl.extensions.llmcache import SemanticCache
cache = SemanticCache(
name="llm_cache",
redis_url="redis://localhost:6379",
distance_threshold=0.15 # cosine distance, lower = more similar
)
# First call: cache miss, store result
response = cache.check(prompt="What is Python?")
if not response:
answer = "Python is a high-level, easy-to-read, versatile programming language..."
cache.store(prompt="What is Python?", response=answer)
print("Cache miss — stored")
else:
print(f"Cache hit: {response[0]['response']}")
# Next time asking "Tell me about Python" → still a cache hit
# because the vector embedding is similar to "What is Python?"That’s all it takes. The next time a user asks “Tell me about Python” or “What kind of language is Python?” — even with different wording — Redis returns the cached result in a few milliseconds instead of making an expensive API call.
The Problem RedisVL Solves
Why Exact-Match Caching Falls Short
Traditional caching uses a hash of the question as the key. “What is Python?” and “What kind of language is Python?” are two completely different keys even though the answer is identical. Result: cache hit rates are very low — often below 5% in practice.
Semantic caching uses vector embeddings instead of hashes. “What is Python?” and “What kind of language is Python?” have very similar embeddings — similar enough to both hit the same cache entry with a threshold of 0.15. Real-world hit rates can reach 35–45%, significantly higher than exact-match caching.
Check if Redis Supports Vector Search
redis-cli MODULE LIST | grep search
# Must see "search" in the output
# If not found → use Redis Stack or Redis Cloud with Search enabledDeep Dive: Custom Vector Search Index
Building a Knowledge Base for RAG
RedisVL goes beyond caching — you can also build a vector search index directly on Redis, which is great for RAG or knowledge base search. I’ve used it to index internal company documents (~3,000 entries), with query latency consistently under 15ms on that dataset:
from redisvl.index import SearchIndex
from redisvl.schema import IndexSchema
import numpy as np
from openai import OpenAI
client = OpenAI()
# Define schema
schema = IndexSchema.from_dict({
"index": {
"name": "docs_index",
"prefix": "doc:",
"storage_type": "hash"
},
"fields": [
{"name": "content", "type": "text"},
{"name": "category", "type": "tag"},
{"name": "embedding", "type": "vector", "attrs": {
"dims": 1536,
"distance_metric": "cosine",
"algorithm": "hnsw",
"datatype": "float32"
}}
]
})
index = SearchIndex(schema, redis_url="redis://localhost:6379")
index.create(overwrite=True)Loading Data and Querying
from redisvl.query import VectorQuery
def get_embedding(text):
res = client.embeddings.create(input=text, model="text-embedding-3-small")
return res.data[0].embedding
docs = [
{"content": "Docker is an application containerization platform", "category": "devops"},
{"content": "Kubernetes orchestrates container deployments", "category": "devops"},
{"content": "Redis is a high-speed in-memory database", "category": "database"},
]
data = []
for i, doc in enumerate(docs):
emb = get_embedding(doc["content"])
data.append({
"content": doc["content"],
"category": doc["category"],
"embedding": np.array(emb, dtype=np.float32).tobytes()
})
index.load(data)
# Semantic search
query_emb = get_embedding("Container orchestration tool")
results = index.query(
VectorQuery(
vector=query_emb,
vector_field_name="embedding",
return_fields=["content", "category"],
num_results=3
)
)
for r in results:
print(f"Score: {r['vector_distance']:.3f} | {r['content']}")Advanced: Metadata Filtering, TTL, and Async
Per-User Context Caching
cache = SemanticCache(
name="app_cache",
redis_url="redis://localhost:6379",
distance_threshold=0.15,
ttl=3600 # Cache expires after 1 hour
)
# Store with metadata
cache.store(
prompt="Explain neural networks",
response="Neural networks are machine learning models...",
metadata={"language": "en", "user_tier": "premium"}
)
# Check with filter — only hit cache for the same language
results = cache.check(
prompt="What is a neural network?",
filter_expression="@language:{en}"
)
if results:
print(f"Cache hit: {results[0]['response']}")Async for High-Traffic Production
from redisvl.extensions.llmcache import AsyncSemanticCache
async def handle_request(prompt: str, llm_func):
cache = AsyncSemanticCache(
name="async_cache",
redis_url="redis://localhost:6379",
distance_threshold=0.15
)
cached = await cache.acheck(prompt=prompt)
if cached:
return cached[0]["response"]
response = await llm_func(prompt)
await cache.astore(prompt=prompt, response=response)
return responsePractical Tips
Choosing the Right distance_threshold
After about a week of production usage with 10,000+ requests, here’s what I observed across different threshold ranges:
- 0.05–0.10: Very strict. Low cache hit rate (~15%) but high response accuracy.
- 0.10–0.20: Good balance. Hit rate ~35–45%, accurate enough for most use cases.
- 0.20+: Aggressive. High hit rate but may return “close enough” results instead of exact matches.
Start with 0.15, monitor for 1–2 weeks, then adjust.
Use an Embedding Model with Multilingual Support
RedisVL defaults to text-embedding-ada-002. To run locally at no cost with better multilingual support:
from redisvl.utils.vectorize import HFTextVectorizer
vectorizer = HFTextVectorizer(
model="sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
# Supports 50+ languages including Vietnamese, runs fully locally
)
cache = SemanticCache(
name="multilingual_cache",
vectorizer=vectorizer,
redis_url="redis://localhost:6379",
distance_threshold=0.15
)LangChain Integration
Already using LangChain? Just add a few lines of config and all LLM calls automatically go through semantic caching:
from langchain_community.cache import RedisSemanticCache
from langchain_openai import OpenAIEmbeddings
import langchain
langchain.llm_cache = RedisSemanticCache(
redis_url="redis://localhost:6379",
embedding=OpenAIEmbeddings(),
score_threshold=0.2
)
# From here, all LLM calls in LangChain automatically use semantic cache
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o-mini")
response = llm.invoke("What is Redis?") # Second call will be a cache hitAvoiding Cache Poisoning
Cache poisoning is a risk that often goes unnoticed. If the LLM hallucinates on the first call, that incorrect response gets cached — and every similar question afterward receives the wrong answer. Preventive measures:
- Set a reasonable TTL (3600–86400 seconds depending on data type)
- Don’t cache questions about real-time data (stock prices, weather)
- Use metadata to tag the source, making it easy to batch-invalidate when needed
- Monitor and have a mechanism to clear cache when system errors are detected:
# View stats
stats = cache.index.info()
print(f"Total cached entries: {stats['num_docs']}")
# Clear all cache when rolling out a new model
cache.clear()Already have Redis in your stack? Adding RedisVL requires no new infrastructure. Cache hits come back in under 10ms versus 1–3 seconds for LLM calls — with a 35–45% hit rate at threshold 0.15, actual token costs drop noticeably after the first few weeks. Worth trying before switching to a dedicated vector DB.
