Caching Strategies

The Problem

Database queries are expensive — 5-50ms each, limited by connection pools, and contention under load. Caching stores frequently accessed data in fast in-memory stores (Redis, Memcached) to serve reads in <1ms. But caching introduces the hardest problem in computer science: cache invalidation. When does the cache become stale? How do you update it? What happens during cache failures? The wrong caching strategy can cause stale data bugs, thundering herds on cache misses, and inconsistency between cache and database that’s impossible to debug.

Architectural Approaches

Architecture

Loading diagram...

Key Components

Data Model

Loading diagram...

Cache Invalidation Patterns

“There are only two hard things in Computer Science: cache invalidation and naming things.” — Phil Karlton

TTL-based expiry: Set a time-to-live on every cache entry. After TTL expires, the next read triggers a cache miss and re-fetch. Simple but creates a staleness window equal to the TTL. Use shorter TTLs (30-60s) for frequently changing data, longer (5-60min) for stable data.

Event-driven invalidation: When data changes, publish an event that triggers cache deletion. Most accurate but requires reliable event delivery. If the invalidation event is lost, the cache stays stale until TTL expires.

Version-based invalidation: Include a version number in the cache key: user:123:v7. When data changes, increment the version. Old cache entries naturally expire via TTL. Never serves stale data but wastes cache space with old versions.

Tag-based invalidation: Tag cache entries with categories. Invalidate all entries with a given tag. Example: tag all product-related caches with product:456. When product 456 changes, invalidate all entries tagged with it. Redis doesn’t support this natively — implement with sets tracking keys per tag.

Stampede prevention: When a popular cache entry expires, hundreds of requests simultaneously hit the database. Solutions:

• Lock-based: First request acquires a lock, fetches from DB, populates cache. Others wait on the lock.
• Probabilistic early expiry: Each request has a small chance of refreshing before TTL. Spreading refreshes over time.
• Background refresh: A background worker refreshes popular entries before they expire.

Multi-Layer Caching

Production systems typically use multiple cache layers, each with different latency and capacity characteristics:

L1 — In-process cache (0.001ms, 100MB):

• Node.js: lru-cache package
• Java: Caffeine or Guava Cache
• Go: groupcache or ristretto
• Best for: configuration, feature flags, frequently accessed immutable data
• Invalidation: TTL + pub/sub notification to all instances

L2 — Distributed cache (0.5-2ms, 10-100GB):

• Redis Cluster or Memcached
• Shared across all application instances
• Best for: session data, user profiles, API response caching
• Invalidation: direct DELETE on write, TTL as safety net

L3 — CDN / Edge cache (0-50ms, terabytes):

• Cloudflare, CloudFront, Fastly
• Best for: static assets, public API responses, HTML pages
• Invalidation: cache tags, surrogate keys, or purge API

Cache hierarchy flow: Request → L1 check → L2 check → L3 check → Origin (database). Each layer handles the miss and populates the cache for the next request. A well-tuned hierarchy achieves 95%+ hit rate at L1+L2, reducing database load dramatically.

Consistency challenge: L1 caches across different instances can be inconsistent (Instance A invalidated, Instance B hasn’t). Use short L1 TTLs (5-15 seconds) and pub/sub invalidation to minimize the inconsistency window.

Cache Key Design

A good cache key strategy prevents collisions, enables efficient invalidation, and makes debugging easier.

Key format: {prefix}:{entity}:{id}:{variant}

• Prefix: service or module name for namespacing
• Entity: the data type (user, product, feed)
• ID: unique identifier
• Variant: optional qualifier (locale, version, view)

Examples:

• api:user:123:profile — User 123’s profile
• api:user:123:feed:page:1 — User 123’s feed, page 1
• api:product:456:en — Product 456 in English
• api:search:q=shoes:page=1:sort=price — Search results cache

Hashing long keys: For complex queries, hash the parameters: api:search:{md5(params)}. Keep a reverse mapping for debugging.

Key expiry patterns:

• User-specific data: 5-15 min TTL (changes on user action)
• Catalog data: 1-24 hour TTL (changes infrequently)
• Computed aggregations: 1-5 min TTL (expensive to recompute)
• Feature flags: 30-60 second TTL (needs freshness)

Avoid hot keys: If one key gets disproportionate traffic (e.g., a viral post), it creates a hotspot on a single Redis shard. Mitigate with key replication (store copies with random suffixes) or L1 caching with very short TTL.

Real-World Examples

Architectural Tradeoffs

# Learn the System Design of: Caching Strategies

You are an expert software architect and teacher. I want to understand caching strategies as a practical system design capability. (Cache-aside, write-through, invalidation patterns, and distributed caching with Redis).

Use the following architectural guide as a learning reference. Avoid one-size-fits-all recommendations. Keep assumptions, costs, and tradeoffs explicit.

---

import { Mermaid, ComponentList, ComponentCard, ApproachList, Approach, Example, Tradeoff } from '@/components/mdx';

## The Problem

Database queries are expensive — 5-50ms each, limited by connection pools, and contention under load. Caching stores frequently accessed data in fast in-memory stores (Redis, Memcached) to serve reads in <1ms. But caching introduces the hardest problem in computer science: cache invalidation. When does the cache become stale? How do you update it? What happens during cache failures? The wrong caching strategy can cause stale data bugs, thundering herds on cache misses, and inconsistency between cache and database that's impossible to debug.

## Architectural Approaches

<ApproachList>
<Approach
  name="Cache-Aside (Lazy Loading)"
  pros={[
    'Simple to implement — application has full control',
    'Only caches data that is actually requested (demand-driven)',
    'Cache failure is graceful — falls back to database',
    'Works with any cache backend (Redis, Memcached, local)',
  ]}
  cons={[
    'First request after invalidation is slow (cache miss → DB query)',
    'Risk of stale data if invalidation fails',
    'Thundering herd: many requests hit the database on popular cache miss',
    'Requires careful TTL tuning per data type',
  ]}
>
The application manages the cache explicitly. On read: check cache first, if miss then query the database and store the result in cache. On write: update the database and invalidate (delete) the cache entry. The next read re-populates the cache. The most common pattern.
<...

[Full prompt copied to clipboard]