Replication is presented as a core technique for low-latency, highly available, and scalable systems. By keeping multiple copies of data close to users, systems can serve requests from nearby replicas while also surviving node or data center failures. The chapter emphasizes the central trade-off: tighter consistency requires more coordination, which increases latency—especially at the tail. It relates replication to colocation and caching, calls out costs such as added storage and write coordination, clarifies the difference between high availability and fault tolerance, and situates replication as a key enabler of scale-out parallelism.
Consistency models define what clients observe across replicas. Linearizability delivers a single-copy illusion with higher coordination latency; eventual consistency minimizes write latency but tolerates temporary divergence (often mitigated with strong eventual consistency and read repair); causal and session consistency lie between, preserving cause-before-effect and per-session monotonicity/read-your-writes. Systems must explicitly consider the read-your-writes property. A local-first approach drives latency toward zero by operating on-device and reconciling later, trading off global consistency. The chapter contrasts asynchronous replication, which acknowledges before replicas apply changes (fast but risks loss/inconsistency), with synchronous replication, which waits for replicas or quorums (slower but stronger guarantees).
Three replication strategies shape latency, availability, and complexity: single-leader (simple but a potential bottleneck with failover needs), multi-leader (accepts local writes across sites for lower latency while introducing write conflicts handled via sharding, CRDTs, or operational transforms), and leaderless (any node can accept writes, relying on quorums and often CRDTs, trading strict ordering for scalability and availability). For strong guarantees and fault tolerance, state machine replication via consensus (Paxos, Raft, Viewstamped Replication) uses quorums of 2f+1 to survive f failures and apply the same ordered log across nodes; the chapter explains VSR’s primary-backup views and how quorum acknowledgments and network distance compound latency. It concludes with a minimal primary-replica key-value store that broadcasts updates and uses snapshots, illustrating the practical mechanics and latency/consistency trade-offs in real systems.
An example of a linearizable key-value store with three replicas. Each of the replica is in the same identical state after every operation because linearizability gives the illusion of a single copy of data being accessed and updated.
An example of a eventually consistent key-value store with three replicas.
Example of single leader replication where clients communicate with a single node, the leader, and other nodes in the cluster acts as followers.
Example of multi-leader replication where clients communicate with more than one nodes, the leaders, but there are other nodes in the cluster acting as followers.
Example of leaderless replication where clients talk to all the nodes in the cluster and no node has a special status.
Example of asynchronous replication where client request is acknowledged before replication is completed.
Example of synchronous replication where client request acknowledgement is delayed until replication completes.
Example of viewstamped replication normal operation message flow.
Summary
Replicating data has multiple benefits reducing latency, improving reliability and availability, and helping prevent data loss due to hardware failures, power outages, or other unforeseen circumstances.
Strong consistency (linearizability) is the gold standard of data access, whereas eventual consistency is the weakest consistency model. Between the two, there are other consistency levels such as causal and session consistency with different trade-offs.
Replication can either be based on single-leader, multiple leaders, or be leaderless, depending on the requirements. Replication can also be synchronous or asynchronous, which impacts latency and consistency.
FAQ
What is replication, and how does it help reduce latency?Replication maintains multiple synchronized copies of the same data across nodes and/or data centers. It reduces access latency by serving requests from the nearest copy (similar to CDNs), while also improving availability and scalability by eliminating single points of failure and enabling parallelism.What are the main trade-offs of using replication?- Higher storage costs because you keep multiple copies
- Increased write and/or read latency when stronger consistency requires coordination
- Potential for stale reads and conflicts under weaker models
- Operational complexity (failover, conflict resolution, topology choices)What is strong consistency (linearizability), and how is it different from serializability?Strong consistency (linearizability) makes a system appear as if there is a single copy of the data: every operation takes effect atomically and is immediately visible to all clients. Linearizability is about single operations on single objects. Serializability, by contrast, is a transactional property over multiple operations and possibly multiple objects, ensuring the outcome is equivalent to some serial order of whole transactions.What is eventual consistency, and when is it a good fit?Eventual consistency guarantees replicas converge over time, but they may temporarily diverge due to lag, partitions, or failures. It’s a good fit when low latency and high availability are paramount and applications can tolerate stale or divergent reads (e.g., CDNs, some write-heavy workloads). Reads can be fast but may require techniques like read repair or reading from multiple replicas to find the latest value.What are causal and session consistency, and when should I use them?- Causal consistency preserves the order of operations that are causally related (A happens-before B), often implemented with vector clocks. It lowers coordination costs while maintaining intuitive ordering of dependent actions (e.g., collaborative editing).
- Session consistency guarantees monotonic reads and read-your-writes within a single user session—ideal for user-centric apps like e-commerce carts, but not for multi-user collaborative scenarios.How do single-leader, multi-leader, and leaderless replication differ?- Single-leader: One leader accepts writes; followers replicate. Simpler to reason about, but can bottleneck, has failover downtime risks, and can add latency for remote writers.
- Multi-leader: Multiple leaders accept writes (often across regions). Reduces write latency for distant clients but introduces conflict risks; requires avoidance (e.g., sharding) or resolution (e.g., CRDTs/OT).
- Leaderless: Any node can accept writes; improves write latency, scalability, and availability, but weakens ordering guarantees. Often uses quorum reads/writes and CRDTs; not strongly consistent.What is the difference between asynchronous and synchronous replication?- Asynchronous: Primary acknowledges the client before replicas are updated. Very low write latency and high throughput, but risks stale reads and data loss on failure. Typically eventually consistent.
- Synchronous: Primary waits for replicas (often a quorum) before acknowledging. Stronger consistency (can be linearizable) but higher write latency and lower throughput.What does the read-your-writes property mean, and how can it be violated?Read-your-writes ensures a client immediately sees its own writes. With asynchronous replication, a client might write to one node and then read from another that hasn’t received the update due to replication lag, violating this property. Always check whether your system guarantees read-your-writes and, if needed, route reads appropriately.How does replication improve availability and scalability?Replication enhances availability by keeping data accessible despite node or data center failures, reducing tail latency associated with outages. For scalability, it enables scale-out by distributing load across nodes that hold copies of data. The “number of nines” expresses target availability; higher nines allow very little downtime and usually require more redundancy and careful design.What is state machine replication, and how do consensus protocols (e.g., Raft, Paxos, VSR) affect latency and fault tolerance?State machine replication ensures all replicas apply the same ordered sequence of commands, keeping their state consistent even across failures. Consensus protocols like Paxos, Raft, and Viewstamped Replication (VSR) use majorities (quorums) to order and commit operations. A cluster of 2f+1 replicas tolerates f non-Byzantine failures. Latency includes client-to-leader round-trip plus quorum acknowledgments; spreading replicas across data centers improves availability but increases these latencies. VSR uses a primary-backup model with views and deterministic primary changes, offering a simple mental model for ordering and failover.
pro $24.99 per month
access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
Latency ebook for free
