Real-World Dilemma: The RAM Bottleneck on a 1GB VPS
A few years ago, I took on a batch job to process financial reports on a cheap VPS with just 1GB of RAM. The script’s role was to scrape exchange rates from four partner APIs, reconcile the data, and compute statistical metrics. Each cycle took about 8 to 10 seconds. Most of that latency stemmed from network round-trips and repetitive data aggregations.
The immediate fix back then: caching. I slapped @functools.lru_cache across the processing functions. Initial results looked very promising. Response times plummeted from 8 seconds to under 20ms for data already held in RAM. However, as the data volume grew, the system ran into three fatal issues:
- Data wiped on restart: Every time a patch was deployed or the process restarted, in-memory cache was completely wiped. The system had to shoulder heavy recomputations from scratch.
- Killed by the OOM Killer: The dataset expanded from tens of megabytes to 400MB. The VPS ran completely out of memory. The Linux OOM Killer promptly terminated the Python process without even writing a log warning.
- Worker fragmentation: Scaling to a multi-process architecture with Gunicorn and Celery (4 workers) meant each worker maintained its own isolated chunk of RAM. Just after Worker 1 finished fetching data, Worker 2 would redundantly make the same API call. CPU cycles were wasted by 3 to 4 times.
Why In-Memory Caching Fails in Production Architectures
The issue isn’t bad code. It stems from the architectural limits of main memory:
- Process isolation: Python process memory is isolated by default. Unless you wire up complex
multiprocessing.shared_memorystructures, processes cannot read each other’s cache. - Lack of persistence: RAM lives and dies with the process. Periodic batch jobs running once every two hours must reload everything from scratch if the cache only resides in RAM.
- RAM is far costlier than SSDs: Modern NVMe drives read sequentially at several GB/s and cost dozens of times less than RAM. Stuffing gigabytes of transient data into RAM on low-spec servers is a risky bet.
Three Common Approaches and Their Trade-offs
Before arriving at an optimal solution, most backend engineers cycle through these three approaches:
1. Manually dumping to JSON or Pickle files
This pattern is very familiar: hash the parameters into a filename, then call pickle.dump() to store it under /tmp.
import pickle
import os
def get_cached_report(key):
path = f"/tmp/cache_{key}.pkl"
if os.path.exists(path):
with open(path, "rb") as f:
return pickle.load(f)
# Recompute on cache miss...
The pitfall: Race conditions. Two workers writing to the same file in the exact same millisecond will instantly corrupt the pkl file. You also have to roll your own TTL expiration logic and manually clean up when the disk hits 100% capacity.
2. Deploying Redis or Memcached
Redis is the industry gold standard for caching. High speed, great multi-process support, and robust built-in TTL.
The trade-off: Operational overhead. For internal scripts, lightweight scrapers, or cron jobs, installing a Redis daemon, opening network ports, managing firewall security, and maintaining TCP connections is unnecessary friction. The Redis daemon alone consumes 40-70MB of static RAM out of the box.
3. Building a custom cache wrapper on SQLite
SQLite provides rock-solid ACID transactions. However, designing custom schemas, handling serialization/deserialization for complex objects, and implementing LRU eviction logic requires significant maintenance effort.
The Optimal Solution: DiskCache
DiskCache is a pure-Python library built to solve this exact problem cleanly. Under the hood, DiskCache pairs SQLite in Write-Ahead Logging (WAL) mode for metadata management with direct file writes on disk for larger objects. On modern NVMe drives, DiskCache clocks in at 25,000 to 40,000 reads per second. This can even outperform Redis running over localhost because it completely bypasses TCP socket protocol overhead.
When extracting data from logs before caching, you will often need to normalize strings with regex. If you need to quickly test complex regular expressions without opening a separate test script, check out the regex tester at toolcraft.app/en/tools/developer/regex-tester to validate patterns directly in your browser.
Step 1: Installation
pip install diskcache
Step 2: Basic Cache Usage with Key-Value and TTL
The Cache object operates just like a dictionary, but everything is safely synchronized to disk:
from diskcache import Cache
import time
# Initialize cache in the target directory
cache = Cache('/tmp/my_disk_cache')
# Store key with time-to-live (expire = 60 seconds)
cache.set('user_101', {'name': 'Nguyen Van A', 'role': 'admin'}, expire=60)
# Read data back
user = cache.get('user_101')
print("Cached data:", user)
# Check key existence
if 'user_101' in cache:
print("Key is still valid")
# Check after expiration
time.sleep(61)
print("After 60s:", cache.get('user_101')) # Returns None
Step 3: Optimizing Heavy Functions with @memoize
Instead of manual get/set calls, simply attach @cache.memoize. The library automatically hashes function arguments into cache keys:
from diskcache import Cache
import time
cache = Cache('./api_cache_dir')
@cache.memoize(expire=3600) # Cache results for 1 hour
def fetch_partner_rates(date_str: str):
print(f"[API CALL] Fetching exchange rates for {date_str}...")
time.sleep(2.5) # Simulate network latency
return {"USD": 25450, "EUR": 27300, "date": date_str}
# Call 1: Real execution, takes 2.5s
t0 = time.time()
print(fetch_partner_rates('2026-03-30'))
print(f"Run 1 took: {time.time() - t0:.3f}s")
# Call 2: Read directly from disk, takes under 2ms
t0 = time.time()
print(fetch_partner_rates('2026-03-30'))
print(f"Run 2 took: {time.time() - t0:.3f}s")
Step 4: Ensuring Multi-Process Safety (Process-Safe)
Thanks to SQLite WAL mode and automatic OS-level file locking, multiple child processes can read and write concurrently without race conditions:
from multiprocessing import Pool
from diskcache import Cache
import os
cache = Cache('./shared_cache')
def worker_task(item_id):
# Independent workers safely access the shared cache directory
val = cache.get(item_id)
if val is None:
val = f"Computed_item_{item_id}_by_pid_{os.getpid()}"
cache.set(item_id, val, expire=300)
return val
if __name__ == '__main__':
# 4 concurrent child processes
with Pool(processes=4) as pool:
results = pool.map(worker_task, [10, 20, 30, 10, 20, 30])
print("Results:", results)
Step 5: Enforcing Storage Quotas with size_limit (Automatic Eviction)
To prevent your disk from ever filling up, specify a cap with the size_limit parameter (in bytes). Once exceeded, DiskCache uses a Least Recently Used (LRU) algorithm to evict the oldest entries:
from diskcache import Cache
# Cap cache at 500MB (500 * 1024 * 1024 bytes)
cache = Cache('./bounded_cache', size_limit=524288000)
# DiskCache automatically purges old records when reaching 500MB
When to Use DiskCache vs. Redis?
Pick the right tool based on your infrastructure architecture:
- Choose DiskCache when: Your app runs on a single node, a low-RAM VPS (1-2GB), CLI tools, desktop apps, or cron jobs needing persistent caching across restarts without maintaining a Redis server.
- Choose Redis when: You run a distributed microservices architecture where multiple servers must share a central cache, or when you require advanced primitives like Pub/Sub, Sorted Sets, and Redis Streams.
If your system doesn’t need multi-node scaling yet, DiskCache offers a pragmatic approach: clean setup, rock-solid stability, and maximum RAM savings for your server.
