One of my most popular blogs to date is also my first blog on Write-Ahead Logs. But as with everything in technology, we evolve the existing solutions to become better. WBL — Write-Behind Log Write-behind logging (WBL) is a logging technique used in database management systems (DBMS) to improve performance. It differs from write-ahead…

The Two-Phase Commit protocol is a distributed algorithm that ensures that a transaction is either committed or rolled back consistently across all nodes in a distributed system. The protocol involves two phases, as the name suggests. Phase 1 — Prepare Phase: The coordinator node sends a prepare message to all…

Most systems that we work with on a general basis, require the latest state of the system and we use CRUD operations to keep the system at it’s latest state. However, there are systems that require temporality, which means your system doesn’t just need the latest state, but also can…

One of the tenets of building Distributed Systems is Fault Tolerance. While there are multiple strategies used to achieve fault tolerance in a distributed system, today we’ll talk about the Bulkhead Pattern. Let’s explain the need for a Bulkhead pattern with some examples - Imagine a distributed system that processes…

As we deal with more complex distributed systems, we’ll often come across use cases where we need to atomically perform operations on multiple data sources in a consistent manner. So, let’s assume that we are persisting order data into an RDBMS. The ML team might want to perform some analytics…

In my last article, we talked about how Centralized Coordination can help us reduce costs associated with performing Linearizable operations in a huge cluster. The idea is to perform operations requiring Linearizability in a small cluster(3–5 nodes). However, just choosing a small cluster doesn’t solve our problem completely. …

The beauty of distributed systems is that you can scale horizontally by adding more compute/resources to your system. …

Ensuring consistency of critical data for which we want to have exclusive access, can be achieved through locking. However, in a distributed setup, we can run into scenarios where because of a network partition, process pauses or nodes go down. This leads to the resources being locked indefinitely! Leases To overcome…

Through my last series about Clocks & Version Vectors, the idea was to discuss problems establishing order/causality in a distributed system and ways to achieve the same. One problem in the real world, that needs both concepts is Distributed Snapshots! Problem Statement Point in Time Snapshots is critical for capturing the “consistent”…