Video Transcoding Pipeline

The Problem

Users upload videos in countless formats (MP4, MOV, AVI, WebM) recorded on different devices with varying resolutions, codecs, and bitrates. To deliver a smooth playback experience across all devices and network conditions, videos must be transcoded into multiple formats and quality levels. Adaptive bitrate streaming (ABR) serves the right quality for each viewer’s bandwidth in real time. The transcoding pipeline must handle large files (multi-GB), scale to thousands of concurrent jobs, manage compute costs (transcoding is CPU/GPU intensive), and deliver results within minutes — not hours.

Architectural Approaches

Architecture

Loading diagram...

Key Components

Data Model

Loading diagram...

Adaptive Bitrate Streaming

ABR serves multiple quality levels and lets the player switch dynamically based on network conditions.

HLS (HTTP Live Streaming): Apple’s protocol, supported everywhere. Master playlist (.m3u8) references variant playlists for each quality level. Each variant is split into 2-10 second .ts segments. Player downloads segments sequentially, switching quality between segments based on bandwidth.

DASH (Dynamic Adaptive Streaming over HTTP): Open standard (ISO). Similar concept with an MPD manifest and segmented media files. Better codec flexibility (supports VP9, AV1 natively). Less universal than HLS on Apple devices.

Common output ladder (quality levels):

• 1080p @ 5000 kbps (high quality, good network)
• 720p @ 2500 kbps (balanced)
• 480p @ 1000 kbps (mobile / slow network)
• 360p @ 600 kbps (very slow network)
• Audio-only @ 128 kbps (extremely poor network)

Segment duration: 2-6 seconds is typical. Shorter segments enable faster quality switching but increase manifest size and request count. 4 seconds is a common default.

Per-title encoding: Netflix pioneered analyzing each video’s complexity to choose optimal bitrates. A simple animation needs less bitrate than an action scene. Per-title encoding reduces storage and bandwidth by 20-40% without quality loss.

Codec Selection and Tradeoffs

H.264 (AVC): The universal codec. Supported by every device and browser. Good quality-to-size ratio. Patent-licensed but widely available. Use as the baseline — always output H.264.

H.265 (HEVC): 30-50% better compression than H.264 at the same quality. Supported on modern devices (iOS, recent Android, Safari). Complex patent licensing. Use for 4K content where bandwidth savings are significant.

VP9: Google’s royalty-free codec. Similar compression to H.265. Supported in Chrome, Firefox, Android. Not supported in Safari/iOS. Use as an alternative to H.265 for web delivery.

AV1: Next-generation royalty-free codec (Alliance for Open Media). 30% better than H.265 but very slow to encode (10-100x slower than H.264). Hardware decoding is becoming available. Use for pre-encoded content where encoding time is not a constraint.

Encoding speed vs quality:

• FFmpeg presets: ultrafast → superfast → veryfast → faster → fast → medium → slow → slower → veryslow
• “medium” is the default — good balance of speed and quality
• “fast” for time-sensitive content (live events, user uploads)
• “slow” for premium content worth the extra encoding time

Two-pass encoding: First pass analyzes the video, second pass encodes with optimal bit allocation. 50-100% slower but better quality for the same bitrate. Use for premium content.

Cost Optimization

Video transcoding is one of the most compute-intensive workloads. Cost optimization is critical.

Spot/preemptible instances: Transcoding is fault-tolerant (can restart from checkpoints). Use AWS Spot Instances (60-90% cheaper) or GCP Preemptible VMs. Implement checkpoint/resume for long-running jobs in case of instance termination.

GPU vs CPU encoding: NVIDIA NVENC hardware encoding is 5-10x faster than CPU encoding. GPU instances are more expensive per hour but cheaper per video due to speed. Break-even depends on your volume — typically GPU wins above 100 videos/day.

Encode only what’s needed: Don’t transcode to 1080p if the source is 720p. Analyze input resolution and only generate output profiles at or below the source quality. Skip audio transcoding if the input codec is already AAC.

Just-in-time transcoding: For low-traffic content, don’t pre-transcode all quality levels. Transcode the most common profiles (720p, 480p) immediately, and transcode higher profiles on first request. Cache the result for subsequent viewers.

Storage optimization: Delete intermediate files after packaging. Keep only the final HLS/DASH segments. For rarely accessed content, use S3 Intelligent-Tiering to automatically move to cheaper storage classes.

Encoding efficiency metrics: Track cost per minute of output video, encoding speed ratio (real-time = 1x, 2x = twice real-time speed), and quality metrics (VMAF score) to optimize the cost/quality tradeoff.

Real-World Examples

Architectural Tradeoffs

# Learn the System Design of: Video Transcoding Pipeline

You are an expert software architect and teacher. I want to understand video transcoding pipeline as a practical system design capability. (Adaptive bitrate encoding, queue-based processing, and HLS/DASH delivery for streaming).

Use the following architectural guide as a learning reference. Avoid one-size-fits-all recommendations. Keep assumptions, costs, and tradeoffs explicit.

---

import { Mermaid, ComponentList, ComponentCard, ApproachList, Approach, Example, Tradeoff } from '@/components/mdx';

## The Problem

Users upload videos in countless formats (MP4, MOV, AVI, WebM) recorded on different devices with varying resolutions, codecs, and bitrates. To deliver a smooth playback experience across all devices and network conditions, videos must be transcoded into multiple formats and quality levels. Adaptive bitrate streaming (ABR) serves the right quality for each viewer's bandwidth in real time. The transcoding pipeline must handle large files (multi-GB), scale to thousands of concurrent jobs, manage compute costs (transcoding is CPU/GPU intensive), and deliver results within minutes — not hours.

## Architectural Approaches

<ApproachList>
<Approach
  name="Queue-Based FFmpeg Workers"
  pros={[
    'Full control over encoding parameters and pipeline',
    'FFmpeg supports virtually every codec and format',
    'Cost-efficient — use spot/preemptible instances for workers',
    'Can optimize for quality, speed, or file size per use case',
  ]}
  cons={[
    'Must manage FFmpeg installation, updates, and licensing (codec patents)',
    'Scaling and infrastructure management is your responsibility',
    'Long-running jobs (hours for 4K video) need checkpoint/resume',
    'Monitoring transcoding quality requires expertise',
  ]}
>
Upload triggers a job in a message queue. Worker instances pull jobs, transcode using **FFmpeg** (the industry-standard tool), and upload results to storage. Workers scale horizon...

[Full prompt copied to clipboard]