Implementing the Saga Pattern in Microservices with Node.js: Managing Distributed Transactions

Development tutorial - IT technology blog
Development tutorial - IT technology blog

Quick start: Build a working Saga Orchestrator in 5 minutes

Breaking down a monolith into microservices always brings a tough challenge: data distribution. Imagine a customer places an order, the Payment Service deducts $25, but the Inventory Service reports the item is completely out of stock. At this point, the ACID guarantees of a single database no longer apply, leaving you to handle the refund manually.

To understand how to solve this, let’s build an Orchestration Saga workflow using vanilla Node.js. Create a file named saga-demo.js:

// saga-demo.js
class OrderSagaOrchestrator {
  constructor() {
    this.steps = [];
  }

  addStep({ name, execute, compensate }) {
    this.steps.push({ name, execute, compensate });
    return this;
  }

  async run(context) {
    const executedSteps = [];
    console.log(`[START] Processing Saga for Order #${context.orderId}`);

    for (const step of this.steps) {
      try {
        console.log(`-> Executing: ${step.name}`);
        await step.execute(context);
        executedSteps.push(step);
      } catch (error) {
        console.error(`<!> Error at [${step.name}]: ${error.message}`);
        await this.rollback(executedSteps, context);
        throw new Error(`Saga failed: ${error.message}`);
      }
    }
    console.log(`[DONE] Distributed transaction completed successfully!`);
  }

  async rollback(executedSteps, context) {
    console.log(`[ROLLBACK] Triggering compensating transactions...`);
    // Reverse the list of previously succeeded steps
    const stepsToRevert = [...executedSteps].reverse();
    for (const step of stepsToRevert) {
      try {
        console.log(`<- Reverting: ${step.name}`);
        await step.compensate(context);
      } catch (compensateError) {
        // Log critical error if compensation fails
        console.error(`[CRITICAL] Rollback [${step.name}] failed: ${compensateError.message}`);
      }
    }
  }
}

// Simulate independent Microservices via API calls
const PaymentService = {
  charge: async (ctx) => { console.log(`   [Payment] Deducted $${ctx.amount} successfully.`); },
  refund: async (ctx) => { console.log(`   [Payment] Refunded $${ctx.amount} to user wallet.`); }
};

const InventoryService = {
  reserve: async (ctx) => {
    if (ctx.item === 'out-of-stock-item') {
      throw new Error('Item SKU-9901 is out of stock!');
    }
    console.log(`   [Inventory] Reserved item successfully: ${ctx.item}.`);
  },
  release: async (ctx) => { console.log(`   [Inventory] Released inventory count for item: ${ctx.item}.`); }
};

// Scenario: Customer orders an out-of-stock product
async function main() {
  const saga = new OrderSagaOrchestrator();

  saga
    .addStep({
      name: 'Process payment',
      execute: (ctx) => PaymentService.charge(ctx),
      compensate: (ctx) => PaymentService.refund(ctx)
    })
    .addStep({
      name: 'Reserve inventory',
      execute: (ctx) => InventoryService.reserve(ctx),
      compensate: (ctx) => InventoryService.release(ctx)
    });

  try {
    await saga.run({
      orderId: 'ORD-9921',
      amount: 150,
      item: 'out-of-stock-item'
    });
  } catch (err) {
    console.log(`Result: Order cancelled and refunded safely.`);
  }
}

main();

Run the file in your terminal:

node saga-demo.js

The output clearly shows the execution sequence: Deduct payment → Inventory throws an error → Trigger refund (Compensate). System state returns to its initial consistent state—all without relying on heavy two-phase commit (2PC) locks.

The Technical Mechanics of the Saga Pattern

The Database-per-Service Barrier

Each microservice manages its own isolated database. When a user buys a MacBook Pro, the data flow immediately spans three separate databases: OrderDB (PostgreSQL), PaymentDB (MySQL), and InventoryDB (MongoDB).

Historically, the Two-Phase Commit (2PC) protocol was widely used to lock resources across multiple databases. But on distributed cloud infrastructure, 2PC reveals critical flaws. Response times can spike from 20ms to 2000ms due to network latency alone. Database tables become hard-locked, quickly exhausting connection pools across the entire system.

How Saga Works

Saga tackles this challenge by breaking the global transaction into a sequence of independent Local Transactions. Each service operates solely on its local database, commits immediately, and triggers the next step.

The cornerstone of Saga is the Compensating Transaction. If step N fails, you cannot execute a SQL ROLLBACK to turn back time because steps 1 through N-1 have already committed. Instead, the orchestrator triggers compensating actions in reverse order from N-1 down to 1 (e.g., cancelling the order, issuing a refund via the payment gateway).

Choreography vs. Orchestration

  • Choreography (Event-driven): Services communicate asynchronously via message brokers (such as Kafka or RabbitMQ). When Service A finishes processing, it emits an OrderCreated event. Service B consumes the event, charges the customer, and emits PaymentCompleted. This pattern requires no central coordinator and runs smoothly for short flows (2–3 steps). However, as systems grow to 8–10 microservices, tracking event flows can turn into a tangled mess.
  • Orchestration (Centralized Coordinator): A dedicated service acts as the “conductor.” It sends commands directly to each worker service, waits for responses, and decides whether to execute the next step or initiate compensation. The biggest advantages are straightforward debugging, centralized logging, and real-time order state visibility.

Handling Network Failures, Idempotency, and State Persistence

The sample script works fine locally. However, when deployed to a real Kubernetes cluster, transient network drops are inevitable.

Designing an Idempotent Receiver (Preventing Duplicate Charges)

Consider this scenario: The Orchestrator calls Payment Service to charge a customer. The payment database records the transaction successfully, but the HTTP 200 response times out after 5,000ms. The Orchestrator assumes the request failed and retries. Without an Idempotency mechanism, the customer will be charged twice.

// Idempotency Key validation middleware in Payment Service
async function processPayment(req, res) {
  const { idempotencyKey, orderId, amount } = req.body;

  // 1. Check if the transaction has already been processed
  const existingTx = await db.paymentTransactions.findOne({ idempotencyKey });
  if (existingTx) {
    return res.status(200).json(existingTx.responsePayload);
  }

  try {
    // 2. Charge and persist the idempotency key within the same local DB transaction
    const result = await db.transaction(async (trx) => {
      const payment = await trx.payments.insert({ orderId, amount, status: 'SUCCESS' });
      await trx.idempotencyRecords.insert({
        idempotencyKey,
        responsePayload: payment
      });
      return payment;
    });
    return res.status(201).json(result);
  } catch (err) {
    // Catch unique constraint violations if two concurrent requests hit within the same millisecond
    if (err.code === 'ER_DUP_ENTRY' || err.code === '23505') {
      const duplicateTx = await db.paymentTransactions.findOne({ idempotencyKey });
      return res.status(200).json(duplicateTx.responsePayload);
    }
    return res.status(500).json({ error: 'Transaction processing error' });
  }
}

Persisting Saga State in the Database

The Orchestrator server can crash or be terminated by Kubernetes at any time. To allow newly spawned pods to know whether to proceed or roll back, you must persist each step’s state to a database:

CREATE TABLE saga_instances (
  id VARCHAR(64) PRIMARY KEY,
  saga_type VARCHAR(50) NOT NULL,
  current_step VARCHAR(50) NOT NULL,
  status VARCHAR(20) NOT NULL, -- PENDING, COMPENSATING, COMPLETED, FAILED
  payload JSONB NOT NULL,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

Before dispatching a command to the next service, the Orchestrator updates the record with the new step. A background worker periodically scans records stuck in the PENDING state for over 60 seconds to decide whether to trigger a retry or initiate compensation.

Production Best Practices for Running Sagas

Operating Sagas in production requires rigorous logging and edge-case error handling. Key best practices include:

  • Standardize payload formatting: Data exchanged across HTTP requests or Kafka topics can grow bulky and difficult to inspect during debugging. You can leverage online formatting tools at toolcraft.app (such as toolcraft.app/en/tools/developer/json-formatter) to quickly inspect JSON structures, validate schemas, or test regex expressions without cluttering your environment with extensions.
  • Trace with Trace Context: Attach a unique identifier (UUID) via an x-correlation-id header or the W3C traceparent standard right from the API Gateway. When a compensating transaction fails, you can simply search this ID in Grafana Loki or Datadog to trace the entire execution path across all services.
  • Compensating actions must never fail: When writing refund or inventory-release logic, never introduce business validations that could throw unhandled exceptions. If a third-party dependency (like a banking payment gateway) is down for maintenance, push the message to a Dead Letter Queue (DLQ) with exponential backoff retries, and instantly dispatch an alert to Slack for on-call engineers.
  • Leverage specialized frameworks: Hand-rolling a state machine from scratch is fine for demos or simple workflows. As systems scale, maintaining custom timeouts and retry mechanisms becomes an operational burden. Consider battle-tested solutions such as Temporal.io, Cadence, or Redis-backed BullMQ.
Share: