URL Shortener

The Problem

URL shorteners transform long URLs into compact links (e.g., bit.ly/abc123) for sharing on social media, SMS, and print materials. Beyond basic shortening, production systems need custom short codes, branded domains, click analytics (who clicked, when, where), link expiration, and abuse prevention. The redirect service must handle millions of lookups per second with sub-10ms latency — every millisecond of redirect delay degrades user experience. The system must also prevent abuse (phishing, malware distribution) and scale the analytics pipeline to process billions of click events.

Architectural Approaches

Architecture

Loading diagram...

Key Components

Data Model

Loading diagram...

Short Code Generation Strategy

A 6-character Base62 code supports 62^6 = 56.8 billion unique URLs. A 7-character code supports 3.5 trillion. For most applications, 6 characters is sufficient.

Base62 encoding of distributed ID:

1. Generate a unique 64-bit ID using Snowflake, ULID, or a database sequence
2. Encode in Base62: repeatedly divide by 62 and map remainders to characters
3. Pad to minimum length (6 characters)

Avoiding predictability: If using sequential IDs, add a shuffle step. XOR the ID with a secret key before Base62 encoding. This makes codes appear random while remaining deterministic and reversible.

Custom short codes: Users can request vanity URLs (e.g., /launch-2024). Validate: minimum 4 characters, alphanumeric + hyphens, not in the reserved words list, not already taken. Store custom codes in the same table with a flag.

Collision handling for random codes: With 6-character random codes and 1 billion existing URLs, collision probability per generation is ~1.8%. Use a retry loop (generate, check, retry if exists). After 3 collisions, increase code length by 1 character. In practice, collisions are rare and retries are fast.

Reserved codes: Block codes that conflict with your routes: /api, /login, /admin, /health. Also block offensive words using a filter list.

301 vs 302 Redirects

The HTTP redirect status code has significant implications.

301 Moved Permanently:

• Browser caches the redirect — subsequent visits don’t hit your server
• Better for SEO — passes link equity to the target URL
• Click analytics are incomplete — cached redirects are invisible
• Use for: permanent links where SEO matters and analytics don’t

302 Found (Temporary Redirect):

• Browser does NOT cache — every visit hits your server
• Every click is tracked in your analytics
• Slightly higher latency (extra server round-trip)
• Use for: links where click tracking is important (most URL shorteners)

307/308: Preserve the HTTP method (POST stays POST). Rarely needed for URL shorteners.

Most URL shorteners use 301 for simplicity and SEO, accepting that some clicks won’t be tracked. Analytics-focused shorteners use 302 to capture every click.

Hybrid approach: Use 301 for the initial redirect but embed a tracking pixel or JavaScript redirect on the landing page for analytics. This gives SEO benefits of 301 while still capturing some analytics data.

Performance: The redirect handler must respond in <10ms. Any slower and users perceive a delay when clicking links. Cache hot URLs in Redis (0.1ms lookup) and use connection pooling for database fallback.

Analytics at Scale

A popular URL shortener processes billions of clicks per day. The analytics pipeline must handle this volume without impacting redirect latency.

Async event collection: The redirect handler emits a click event to a queue (Kafka, SQS) and responds immediately. Never block the redirect on analytics processing. If the queue is unavailable, drop the event (redirect availability > analytics completeness).

Click event enrichment: The event processor enriches raw events:

• IP → geolocation (country, city) using MaxMind GeoIP
• User-Agent → device type, browser, OS using ua-parser
• Referrer → categorized source (social, search, direct, email)

Storage: ClickHouse or TimescaleDB for time-series click data. Partition by date for efficient range queries. Pre-aggregate hourly/daily counts for dashboard performance. Keep raw events for 90 days, aggregated data for 2+ years.

Real-time counters: For displaying live click counts, use Redis INCR on the short code key. Periodically flush to the database. This avoids hitting the analytics DB for simple count queries.

Unique visitor counting: Use HyperLogLog (Redis PFADD) for approximate unique visitor counts. Memory-efficient (12KB per counter regardless of cardinality) and accurate to ~0.81% error. Good enough for analytics dashboards.

Bot filtering: A significant portion of clicks are bots (crawlers, preview generators). Filter by user-agent patterns, known bot IP ranges, and behavioral signals (clicks within milliseconds of creation). Show “human clicks” and “total clicks” separately.

Real-World Examples

Architectural Tradeoffs

# Learn the System Design of: URL Shortener

You are an expert software architect and teacher. I want to understand url shortener as a practical system design capability. (Short link generation, redirect resolution, and click analytics at scale).

Use the following architectural guide as a learning reference. Avoid one-size-fits-all recommendations. Keep assumptions, costs, and tradeoffs explicit.

---

import { Mermaid, ComponentList, ComponentCard, ApproachList, Approach, Example, Tradeoff } from '@/components/mdx';

## The Problem

URL shorteners transform long URLs into compact links (e.g., bit.ly/abc123) for sharing on social media, SMS, and print materials. Beyond basic shortening, production systems need custom short codes, branded domains, click analytics (who clicked, when, where), link expiration, and abuse prevention. The redirect service must handle millions of lookups per second with sub-10ms latency — every millisecond of redirect delay degrades user experience. The system must also prevent abuse (phishing, malware distribution) and scale the analytics pipeline to process billions of click events.

## Architectural Approaches

<ApproachList>
<Approach
  name="Base62 Encoding of Auto-Increment ID"
  pros={[
    'Guaranteed unique — no collision checking needed',
    'Compact codes — 6 characters support 56 billion URLs',
    'Deterministic — same ID always produces same code',
    'Sortable — codes are roughly time-ordered',
  ]}
  cons={[
    'Sequential codes are predictable — users can enumerate URLs',
    'Requires a centralized or distributed counter',
    'Short codes for early IDs (1 → "1") may look odd',
    'No customization — codes are assigned, not chosen',
  ]}
>
Generate a unique integer ID (auto-increment counter or distributed ID generator) and encode it in Base62 (a-z, A-Z, 0-9) to produce a short alphanumeric code. ID 1000000 → "4c92". Simple, deterministic, and guaranteed unique.
</Approach>
<Approach
  name="Random Short Code with Col...

[Full prompt copied to clipboard]