Storage Systems: Dropbox/Google Drive, Distributed Email & Distributed File System
Three case studies about keeping user bytes safe at petabyte scale: file sync with block-level dedup and delta sync, a Gmail-scale email service, and a GFS/HDFS-style distributed file system with leases and replication.
About 140 minutes · 15 exam questions · 9 interview questions · 2 code labs
What you will learn
- Three storage designs and how they differ
- Dropbox / Google Drive (1/4): requirements, estimation, API and data model
- Dropbox / Google Drive (2/4): high-level architecture and the upload/download flow
- Dropbox / Google Drive (3/4): chunking, dedup and delta sync
- Dropbox / Google Drive (4/4): conflicts, notifications, versioning and trade-offs
- Distributed email (1/3): requirements, estimation and protocols
- Distributed email (2/3): architecture, send flow and receive flow
- Distributed email (3/3): metadata data model, search, consistency and deliverability
- Distributed file system (1/3): GFS/HDFS architecture and master metadata
- Distributed file system (2/3): leases, the write pipeline and consistency
- Distributed file system (3/3): failure recovery, placement, erasure coding and master HA
- Wrap-up: trade-offs and how to close each design
- Level bar: what mid, senior and staff answers include
- Deep-dive ladders: bad → good → great
Previous: Commerce & Money: Hotel/Ticket Booking, E-commerce, Payment System & Digital Wallet · Next: Advanced: Leaderboard, Stock Exchange, Distributed Cache, Global Rate Limiter, Search Engine/Typeahead, Ad Serving, Recommendation System & Multi-tenant SaaS