The video-on-demand module — replays, highlights, fan uploads, a Netflix/YouTube inside your
sports service. This chapter follows a video from the moment it lands in a bucket through a stateless processing
workflow, out to edge caches worldwide, and into a data model that has to store a web of relationships between
users, channels, and videos.
🧭
How to read this
Two halves. First the data plane — how a video moves: upload → workflow → transcode → CDN →
viewer, plus the storage estimates that justify every design choice. Then the data model — a
long-running debate the chapter frames as "relationships live better in SQL than NoSQL," worked through both a
relational schema and MongoDB's embedded vs. referenced models. Anything marked [+]
is context I added; everything else is rewritten from the five teaching lessons.
01
What it must do, and the workflow that does it
Four functional requirements, YouTube-style: (1) users upload videos via their channel (fan
clips from the stadium/sports bar, plus official broadcaster channels); (2) watch a video from
the VOD home page; (3) follow a channel and get notified of new uploads; (4)
search the platform. Non-functional requirements — scalable, available, resilient — are a given for every
service (a design that doesn't scale isn't a good design), stated explicitly only when special.
🎬
VOD vs. live stream: one key difference
Live streams arrive at a standard resolution. User uploads come in every format, resolution, and
dimension — so you need transcoding rules keyed to each video's metadata. A 144p clip
needs no transcoding; a 4K video fans out into several lower resolutions; 15–30s shorts (Reels/YouTube
Shorts) may get their own handling.
Two buckets, one workflow
Use two object-store buckets: a source bucket where uploads land temporarily, and a
destination bucket where transcoded videos end up. (One bucket works too, but two is cleaner.)
A video landing in the source bucket triggers a workflow.
🔀
What is a workflow?
A DAG — a Directed Acyclic Graph of tasks, with directed edges and no cycles — used to
automate processes, typically via message queues. It gives you an entry point plus failure handling,
retries, rollbacks, parallel execution, and observability. Apache Airflow is a common tool
for defining them.
02
Three stateless functions fired by the DAG
When a video hits the source bucket, the workflow triggers serverless functions:
Metadata extraction — pulls resolution, frame width/height, length, bitrate, audio sample rate,
etc., and writes them to a relational VideoMetadata table (VideoId, Name, Title, Description,
UploadDate, ChannelId, SportsCategory, VideoLikes, Views, VideoSize, Resolution, DiskLocation, …).
Thumbnail generator (runs in parallel) — creates a thumbnail, stores the file in a
VideoThumbnailBucket, and adds its path to a new ThumbnailDiskLocation column. One
thumbnail per video keeps it simple; multiple would need a separate Thumbnail table
(one-to-many).
Transcoding — pulls the video from the source bucket, creates the transcoded versions, and
stores them in the destination bucket.
Figure 1 — The upload workflow: a bucket event triggers a DAG that fans out to parallel
stateless functions, then transcodes source → destination. Original diagram after Figures 5.0–5.3.
💸
Why serverless? Because the work is stateless
Extracting metadata, generating a thumbnail, or transcoding needs nothing beyond the video itself
— not the uploader's identity or history. Stateless events mean instances don't idle holding state; they
spin up only when a video needs processing. That saves money. (Later workflow steps: populate the CDN, push
to viewers' home pages, and send notifications.)
03
Delivery: replicate to the edge, cache in layers
Videos from the destination bucket are intelligently replicated to edge locations — a video
popular in a region goes to that region's edges. Replicating every popular video to every edge globally would be
overkill.
A viewer clicks a video; the request routes to the nearest edge location.
Edge checks its cache. Hit → return to viewer.
Miss → fetch from origin, cache it if it's popular enough per business rules, and
return to the viewer.
🗂️
A second, regional cache layer
Add a regional cloud cache between the edge and the origin. It holds videos less
popular than edge content; as edge videos cool off over time, they demote to this secondary cache. It
significantly cuts the requests reaching the origin data center.
Figure 2 — Layered delivery: edge cache → regional cache → origin, with videos demoting as
they cool. Original diagram after Figure 5.4.
04
The numbers: 900 PB streamed, 212 TB uploaded — daily
The exercise isn't about exact figures — it's about how you reason to them. Assume 300M
subscribers, 100M daily active, each watching ~3 hours/day.
Daily bandwidth consumed
At HD 1080p ≈ 3 GB/hour → 9 GB per viewer/day × 100M viewers = 900 Petabytes/day streamed.
Daily uploads & storage
Assume an upload:consumer ratio of 1:1000 → 100K uploads/day. Split them 10% 4K / 80% 1080p / 10% shorts, and
add every transcoded version:
Thumbnails and metadata are omitted as negligible. Resolution/bitrate references come from Netflix (4K = 7
GB/hr, 1080p = 3 GB/hr, 720p = 1 GB/hr, 480p = 0.7 GB/hr). Without historical data, any such estimate is rough —
the method is the lesson.
05
Modeling the user side (relationally)
Why relational, not a document DB like MongoDB? Because the data is deeply relational: a user
owns multiple channels, subscribes to multiple channels, a video has many comments and thumbnails. Relational
databases specialize in relationships. The schema:
UserStores user profile data.
UserChannelChannels owned by a user. One-to-many from User (UserId as
foreign key).
ChannelA standalone entity — so a channel's details can be read without going
through User.
ChannelSubscriptionBridge table between User and Channel, establishing the
many-to-many following relationship.
A user relates to channels two ways: ownership (User → UserChannel, one-to-many) and
following (User ↔ Channel, many-to-many via the ChannelSubscription bridge). And
Channel has a one-to-many with VideoMetadata (ChannelId as foreign key), since a
channel holds many videos.
Figure 3 — The relational model: ownership (1:many), following (many:many via a bridge), and
channel→videos (1:many). Original diagram after Figures 5.5–5.8.
Ideally a graph model fits many-to-many best (covered in a later chapter), but here it's done
relationally. Which raises the question the next lesson tackles: can a document DB store these relationships
too?
06
Relationships in a document DB: embedded vs. referenced
In a document DB (MongoDB), a document = a relational row and a collection = a
table. It stores relationships two ways. Take the many-to-many User ↔ Channel:
Model
How it works
Trade-off
Embedded
Each Channel document embeds all its subscribers; each User document embeds all channels they
subscribe to.
Highly denormalized — huge duplicate data (a user with 100 channels × 300M users),
and writes must update many places, risking inconsistency. But reads fetch everything in one
query (no join).
Referenced
Separate User, Channel, and ChannelSubscription collections with references (foreign keys) — mirrors
the relational design.
Normalized. Fetching across collections needs extra queries, just like relational
joins — so performance ends up equivalent to the relational approach.
⚖️
When embedding actually wins
Embedding is wrong for the many-to-many subscription case, but great for a one-to-many
like User → UserChannel (channels a user owns): embed that list in the User document and you avoid a join
entirely. In NoSQL, you model data around the queries you expect — which usually ends in
denormalization, precisely to cut read latency. That's the whole reason you'd reach for NoSQL.
[+] The relational world does the reverse: normalize first, then optimize reads with
minimal joins and query caching. But nothing forbids denormalizing in SQL too — if it cuts latency while
preserving integrity, do it.
07
So which database wins?
🏆
The verdict: relational, for relationships
NoSQL databases support relationships but aren't built for them — they're built to access
data with minimum latency, which works best when data is denormalized. A relational database
specializes in relationships, has decades of maturity behind it, and gives strong
consistency — imperative for financial transactions. When dealing with relationships, stick with relational.
(The same relational store will back the subscription-payment module in a later chapter.)
With the upload-and-delivery workflow designed and the data modeled, the next chapter recommends videos to
viewers on the VOD home page via a video feed service.
08
Recap
RequirementsUpload via channel, watch from home page, follow + notify, search.
Scalable/available/resilient are always assumed.
Two buckets + DAGSource → destination buckets; an upload event triggers a
workflow (DAG) — retries, rollbacks, parallelism, observability. Airflow-style.
Stateless functionsMetadata extraction, thumbnail generation (parallel),
transcoding — serverless because the work needs no state, saving money.
Layered edge deliveryIntelligently replicate to regional edges; edge cache →
regional cache → origin, with videos demoting as they cool.
Data modelRelational schema (User, UserChannel, Channel, ChannelSubscription
bridge). Embedded vs. referenced in MongoDB. Relational wins for relationships.
💡
The single idea underneath it all
A VOD system is two pipelines that never touch. The bytes pipeline — the actual video —
wants to be dumb, stateless, and cached as close to the viewer as possible; you push it out to the edge and
let serverless functions chew through it on the way. The relationships pipeline — who owns
what, who follows whom — wants structure and consistency, so it lives in a relational database. Confusing
the two (video files in a database, or relationships in a denormalized document store) is where these
designs go wrong. Keep the heavy dumb data at the edge and the connected smart data in SQL.
09
Further reading
All three external references resolve live. The author's own course cross-links are omitted.