Storage Systems: Dropbox/Google Drive, Distributed Email & Distributed File System

Three case studies about keeping user bytes safe at petabyte scale: file sync with block-level dedup and delta sync, a Gmail-scale email service, and a GFS/HDFS-style distributed file system with leases and replication.

About 140 minutes · 15 exam questions · 9 interview questions · 2 code labs

What you will learn

  1. Three storage designs and how they differ
  2. Dropbox / Google Drive (1/4): requirements, estimation, API and data model
  3. Dropbox / Google Drive (2/4): high-level architecture and the upload/download flow
  4. Dropbox / Google Drive (3/4): chunking, dedup and delta sync
  5. Dropbox / Google Drive (4/4): conflicts, notifications, versioning and trade-offs
  6. Distributed email (1/3): requirements, estimation and protocols
  7. Distributed email (2/3): architecture, send flow and receive flow
  8. Distributed email (3/3): metadata data model, search, consistency and deliverability
  9. Distributed file system (1/3): GFS/HDFS architecture and master metadata
  10. Distributed file system (2/3): leases, the write pipeline and consistency
  11. Distributed file system (3/3): failure recovery, placement, erasure coding and master HA
  12. Wrap-up: trade-offs and how to close each design
  13. Level bar: what mid, senior and staff answers include
  14. Deep-dive ladders: bad → good → great

Previous: Commerce & Money: Hotel/Ticket Booking, E-commerce, Payment System & Digital Wallet · Next: Advanced: Leaderboard, Stock Exchange, Distributed Cache, Global Rate Limiter, Search Engine/Typeahead, Ad Serving, Recommendation System & Multi-tenant SaaS