The Saga Pattern

The Challenge of Distributed Transactions

In a microservice architecture, a key principle is the "Database per Service" pattern, where each service owns its private data. This promotes loose coupling but introduces a significant challenge: how do you maintain data consistency across services? A single business operation might require updates to several databases owned by different services. For example, creating an order in an e-commerce application could involve an Order Service, a Customer Service, and an Inventory Service. If the Customer Service verifies the customer's credit but the Inventory Service finds the product is out of stock, how do you roll back the credit check?

Traditional ACID (Atomicity, Consistency, Isolation, Durability) transactions, which work well in a single database, are not suitable for distributed systems. Two-phase commit (2PC) protocols exist but are often impractical in modern applications due to their blocking nature and the tight coupling they introduce. The Saga pattern provides a solution for managing data consistency across microservices in a non-blocking, asynchronous way.

What is a Saga?

A saga is a sequence of local transactions. Each local transaction is an atomic operation performed by a single microservice that updates its own database. After completing its local transaction, the service publishes an event or sends a command to trigger the next local transaction in the saga. This continues until the entire business process is complete. The key to maintaining consistency is that for every transaction that can fail, there must be a corresponding compensating transaction. A compensating transaction is an operation that undoes the work of a previous transaction. For example, the compensating transaction for "Reserve Credit" would be "Release Credit."

If any local transaction in the saga fails, a series of compensating transactions are executed in reverse order to roll back the work done by the preceding local transactions. This ensures that the system returns to a consistent state, achieving "semantic" atomicity rather than the strict atomicity of an ACID transaction. There are two common ways to coordinate sagas: Choreography and Orchestration.

Diagram: A successful saga (top) and a saga rollback using compensating transactions (bottom).

1. Choreography-based Saga

In a choreography-based saga, there is no central coordinator. Each service participating in the saga is responsible for triggering the next step by publishing an event after it completes its local transaction. Other services subscribe to these events and act accordingly.

How it Works (Order Example):

1. The Order Service receives a `Create Order` request. It creates an order, sets its status to `PENDING`, and publishes an `OrderCreated` event.
2. The Customer Service listens for the `OrderCreated` event. It attempts to reserve credit for the customer. If successful, it publishes a `CreditReserved` event. If not, it publishes a `CreditReservationFailed` event.
3. The Inventory Service listens for the `CreditReserved` event. It checks the inventory for the ordered product. If available, it publishes an `InventoryUpdated` event.
4. The Order Service listens for the `InventoryUpdated` event and changes the order status from `PENDING` to `APPROVED`.

Handling Failures with Choreography:

If the Customer Service cannot reserve credit, it publishes `CreditReservationFailed`. The Order Service listens for this and triggers its compensating transaction: changing the order status to `FAILED`.

Pros:

• Simplicity and Loose Coupling:No central point of control. Services are self-contained and only need to be aware of the events they care about.
• Easy to Implement for Simple Sagas:For sagas with only a few steps, choreography is straightforward to build.

Cons:

• Difficult to Track:It can be hard to determine the status of a business process, as the logic is spread across multiple services. Visualizing the flow requires looking at every participating service.
• Cyclic Dependencies:There's a risk of creating cyclic dependencies between services, where Service A listens to B, B listens to C, and C listens to A.
• Complexity with Many Participants:As the number of steps in a saga grows, the web of events can become very complex and hard to manage.

2. Orchestration-based Saga

In an orchestration-based saga, a central controller, the Orchestrator, is responsible for managing the entire sequence of operations. The orchestrator tells the participating services what to do by sending them commands. It keeps track of the state of the saga and is responsible for triggering compensating transactions in case of a failure.

How it Works (Order Example with Orchestrator):

1. The Order Service receives a `Create Order` request. It creates a new `OrderSagaOrchestrator`.
2. The Orchestrator sends a `ReserveCredit` command to the Customer Service.
3. The Customer Service executes the local transaction and replies to the Orchestrator with `CreditReserved` or `CreditReservationFailed`.
4. If credit was reserved, the Orchestrator sends an `UpdateInventory` command to the Inventory Service.
5. The Inventory Service replies, and if successful, the Orchestrator sends a final `ApproveOrder` command to the Order Service.

Handling Failures with Orchestration:

If at any point a service reports a failure (e.g., the Customer Service replies with `CreditReservationFailed`), the Orchestrator takes control. It knows which transactions have already completed and sends the appropriate compensating commands in reverse order (e.g., if inventory was already updated, it would send a `ReleaseInventory` command).

Pros:

• Centralized Logic:The business process logic is in one place, making it easier to understand, manage, and modify.
• Clear State Management:The orchestrator explicitly manages the state of the saga, so it's always possible to know where a transaction is in its lifecycle.
• Avoids Cyclic Dependencies:Services don't need to know about each other, only about the orchestrator.

Cons:

• Risk of a "Smart" Orchestrator:There's a danger of putting too much business logic into the orchestrator, making it a central monolith and a single point of failure.
• More Infrastructure:Requires implementing and managing the orchestrator component.

Conceptual Code for an Orchestrator

class OrderSagaOrchestrator {
    constructor(orderId, customerId, items) {
        this.orderId = orderId;
        // ... other data
        this.state = 'NEW';
        this.completedSteps = [];
    }

    async start() {
        this.state = 'RESERVING_CREDIT';
        try {
            await customerService.reserveCredit(this.customerId, this.orderTotal);
            this.completedSteps.push('CreditReserved');
            
            this.state = 'UPDATING_INVENTORY';
            await inventoryService.updateInventory(this.items);
            this.completedSteps.push('InventoryUpdated');

            this.state = 'APPROVING_ORDER';
            await orderService.approve(this.orderId);
            this.state = 'COMPLETED';

        } catch (error) {
            this.rollback();
        }
    }

    async rollback() {
        this.state = 'ROLLING_BACK';
        for (const step of this.completedSteps.reverse()) {
            if (step === 'InventoryUpdated') {
                await inventoryService.releaseInventory(this.items);
            }
            if (step === 'CreditReserved') {
                await customerService.releaseCredit(this.customerId, this.orderTotal);
            }
        }
        await orderService.reject(this.orderId);
        this.state = 'FAILED';
    }
}
        

Conclusion

The Saga pattern is an essential tool for managing data consistency in a distributed microservice environment. It provides a robust alternative to traditional distributed transactions by embracing asynchronous communication and eventual consistency. The choice between choreography and orchestration depends on the complexity of the business process. Choreography is simpler for short-lived, simple sagas, while orchestration provides better control and visibility for complex, long-running processes. Successfully implementing sagas requires a shift in thinking away from the immediate consistency of ACID transactions, but it is a necessary step to unlock the scalability and resilience promised by the microservice architecture.