11 Predictive techniques

When you can’t make a slow operation truly faster, you can often hide its delay by doing the work before it’s needed. This chapter frames predictive techniques around a simple pattern: find latency you cannot reduce, predict when results will be required, and initiate the work early so outputs are ready just in time. Approaches like prefetching, optimistic updates, speculative execution, and predictive resource allocation aim to deliver an experience that feels instantaneous. The hard part is prediction: getting it right can create the illusion of zero latency; getting it wrong can waste resources and even make latency worse.

Prefetching hides read latency by retrieving data in advance, using either pattern-based signals (sequential, spatial, or strided access) or semantic cues (dependencies between entities, current context, and per-user history). Done well, it avoids waiting on I/O; done poorly, it wastes bandwidth and pollutes caches. Optimistic updates target write latency by applying changes locally and syncing in the background via a shadow write queue and an optimistic view over local storage. Synchronization then reconciles concurrent edits using strategies such as last-writer-wins with logical or hybrid logical clocks, operational transformation, or CRDTs, with consistency ranging from eventual to causal or strong eventual. Robustness relies on idempotent retries, partial application of changes, rollbacks via snapshots or operation dependencies, and escalation to the UI for human resolution when needed.

Speculative execution generalizes the idea to computing results ahead of time: keep state current with incremental computation (for example, materialized views), explore multiple paths via parallel speculation and keep the fastest useful outcome, and even predict values to provide immediate feedback while confirming the final result. At the infrastructure layer, predictive resource allocation reduces provisioning delays: overprovisioning keeps excess capacity ready for spikes, while pre-warming starts resources just before anticipated demand based on cycles, events, or live trends. All of these methods trade resources, complexity, and potential side effects for speed—mispredictions can displace useful data, steal capacity from real work, or trigger unintended behavior. Used judiciously and measured carefully, predictive techniques are powerful tools to deliver low-latency experiences once simpler optimizations have been exhausted.

With sequential prefetching, we predict that successive data elements relative to the one we’re accessing to are needed in the future. In this example, we access the first data element and, therefore, predict that the next subsequent data elements are also needed.

In spatial prefetching, we make a prediction that nearby data elements are likely to be needed. In this example, access the fourth data element and make a prediction that third and fifth data elements are likely to be needed, and, therefore, prefetch them.

In stride-based prefetching, we make a prediction that data is accessed at fixed intervals. In this example, we access the first data element and make a prediction that every second data element is needed and prefetch them.

Optimistic views build on a base database and a shadow write queue. In this example, we have a database table of tasks with ID, task, and information about completion. The application writes locally to a shadow write queue, which is combined into an optimistic view, shown on left.

Lamport timestamps record the causal relationship between events. Each process maintains a counter, which increments every time the process sends out a message. The process includes the counter as part of the sent out message and when another process receives the message, the process updates its timestamp to the maximum of current and the received timestamp. In this example, you have three processes sending and receiving messages.

Incremental computation updates state incrementally as updates happen. In this example of a materialized view, an INSERT statement to a base table is reflected in a materialized view combining two tables. This ensure that when the view is queried, there is no perceived latency.

With parallel speculation, a backend system can initiate speculative generation of recommendations while the authentication flow is in progress. When login is done, the user can immediately view recommendations if needed. Of course, if the user does not view the recommendations, the speculative work done in parallel is wasted use of compute resources.

• Predictive techniques hide latency by performing operations before they’re needed, effectively creating zero-latency experiences when implemented correctly
• Prefetching retrieves data ahead of time using pattern-based approaches (sequential, spatial, stride-based), semantic methods, or machine learning to predict access patterns
• Optimistic updates hide write latency by allowing immediate client-side changes with background synchronization, requiring conflict resolution mechanisms like timestamps or CRDTs
• Speculative execution performs tasks before confirming they’re needed through techniques like incremental computation, parallel execution paths, and value prediction
• Predictive resource allocation eliminates provisioning latency through overprovisioning (maintaining excess capacity) or pre-warming (just-in-time allocation)
• Failed predictions can waste resources, add complexity, and potentially increase latency, making accurate prediction mechanisms crucial to success
• Each technique trades consistency for latency, requiring careful consideration of application requirements and tolerance for temporary inconsistency

FAQ