Data Systems Theory: System Models, Ordering, Encoding & Derived Data

The theory senior interviews probe without naming it: system models and fault types, safety vs liveness, the two generals and FLP results, state machine replication and why total order broadcast is consensus, Avro schema resolution and rolling upgrades, stream-table duality, the three kinds of stream join, "unbundling the database" with CDC and a log, the end-to-end argument for exactly-once, when Byzantine fault tolerance is worth it, and what modern data systems add.

About 110 minutes · 18 exam questions · 11 interview questions · 2 code labs

What you will learn

  1. System models: timing, node faults and links
  2. Safety vs liveness: the two kinds of correctness property
  3. The two generals problem and FLP: what is impossible, and what still works
  4. State machine replication and total order broadcast (≡ consensus)
  5. Encoding and evolution: Avro writer and reader schemas, and schema registries
  6. Rolling upgrades, "data outlives code" and the three modes of dataflow
  7. Stream–table duality, changelog streams and materialised views
  8. Stream joins: stream-stream, stream-table, table-table, and why joins depend on time
  9. Derived data and "unbundling the database"
  10. The end-to-end argument applied to exactly-once and deduplication
  11. Byzantine fault tolerance: when it actually matters
  12. What modern data systems add that is worth knowing

Previous: Concurrency, Locking & Distributed Locks