Distributed Systems on Software Engineering Learning Paths

Distributed Systems Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

Distributed Systems Fundamentals#

A distributed system is a collection of independent computers that appears to its users as a single coherent system. This deceptively simple definition hides enormous complexity — complexity that has broken production systems at every scale, from startups to hyperscalers.

This section covers why distributed systems are fundamentally harder than single-node systems, the theoretical limits that constrain what is possible, and the models we use to reason about them.

Time & Ordering

Mon, 01 Jan 0001 00:00:00 +0000

Time & Ordering#

In a single-node system, “what happened first?” has a straightforward answer — the CPU’s instruction counter provides a total order. In a distributed system, this question becomes one of the hardest problems in computer science. There is no global clock, no shared counter, and no way to determine the exact order of events across nodes without communication.

This section covers the tools we use to establish ordering in distributed systems — from physical clocks (and why they fail) to logical clocks that capture causality without relying on time at all.

Consistency Models

Mon, 01 Jan 0001 00:00:00 +0000

Consistency Models#

When data is replicated across multiple nodes, different replicas can disagree about the current state. A consistency model defines the rules about what values a read operation can return and when writes become visible. Choosing the right consistency model is one of the most consequential design decisions in a distributed system — it determines the tradeoff between correctness, performance, and availability.

This section covers the hierarchy of consistency models from strongest to weakest, the guarantees each provides, and when to use them.

Consensus Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

Consensus Algorithms#

Consensus is the problem of getting multiple nodes to agree on a single value, even when some nodes crash or messages are lost. It is the foundation of replicated state machines, leader election, distributed locks, and total order broadcast.

The FLP impossibility result (Section 1) proves that deterministic consensus is impossible in a purely asynchronous system. Practical algorithms sidestep this by assuming partial synchrony — the system is eventually timely enough for progress, even if delays occur.

Replication

Mon, 01 Jan 0001 00:00:00 +0000

Replication#

Replication — maintaining copies of data on multiple nodes — serves three goals: fault tolerance (surviving node failures), lower latency (serving reads from nearby replicas), and higher throughput (distributing load). The challenge is keeping replicas consistent while maximising performance and availability.

This section covers the three fundamental replication topologies, the consistency problems each introduces, and the strategies for resolving conflicts.

Single-Leader Replication#

In single-leader (also called primary-backup or master-slave) replication, one node is designated the leader. All writes go to the leader, which replicates changes to followers.

Partitioning & Sharding

Mon, 01 Jan 0001 00:00:00 +0000

Partitioning & Sharding#

When a dataset grows beyond what a single node can store or serve, it must be split across multiple nodes. Partitioning (also called sharding) divides data so each node is responsible for a subset. Combined with replication (Section 5), partitioning enables systems to scale both storage and throughput horizontally.

This section covers the strategies for partitioning data, the mechanisms for distributing partitions across nodes, and the problems that arise when you get it wrong.

Distributed Transactions

Mon, 01 Jan 0001 00:00:00 +0000

Distributed Transactions#

A distributed transaction spans multiple nodes, partitions, or services. It must either commit on all participants or abort on all — partial completion is not acceptable. This is far harder than single-node transactions because any participant can crash, and the network can partition at any point during the protocol.

This section covers the classical protocols (2PC, 3PC), modern approaches (deterministic databases, Spanner’s TrueTime), and the coordination mechanisms that make them work.

CRDTs & Eventual Consistency

Mon, 01 Jan 0001 00:00:00 +0000

CRDTs & Eventual Consistency#

Conflict-free Replicated Data Types (CRDTs) are data structures that can be replicated across multiple nodes, modified independently and concurrently, and merged automatically — always converging to the same state without coordination. They provide strong eventual consistency (Section 3): any two replicas that have received the same set of updates are in the same state, immediately.

This section covers the theory behind CRDTs, the most important CRDT types, and the anti-entropy mechanisms that propagate updates between replicas.

Failure Detection & Recovery

Mon, 01 Jan 0001 00:00:00 +0000

Failure Detection & Recovery#

In a distributed system, detecting whether a remote node has failed is fundamentally uncertain. You cannot distinguish between a crashed node, a slow node, and a partitioned network — all three look the same from the outside: no response. This section covers the mechanisms systems use to detect failures, maintain cluster membership, and recover data after failures.

Note: Application-level resilience patterns (circuit breakers, retries, bulkheads) are covered in System Design: Reliability & Resilience. This section focuses on infrastructure-level failure detection and data recovery mechanisms.

Case Studies

Mon, 01 Jan 0001 00:00:00 +0000

Case Studies#

Theory without practice is incomplete. This section examines how six influential distributed systems apply the concepts from this path. Each case study highlights the key architecture decisions, the tradeoffs accepted, and the lessons learned.

Amazon Dynamo#

Paper: “Dynamo: Amazon’s Highly Available Key-value Store” (DeCandia et al., 2007)

Dynamo was built for Amazon’s shopping cart — a system where availability must never be sacrificed, even at the cost of consistency. It pioneered many ideas that later appeared in Cassandra, Riak, and DynamoDB.