Explained with a diagram · Microservices

Outbox pattern: how the transactional outbox works, with a diagram

In short

The outbox pattern is a way for a service to update its database and publish a message without a distributed transaction. The service stores the outgoing message in an outbox table as part of the same local transaction that changes its business data, and a separate message relay then reads the outbox and sends each message to the message broker. A message is therefore sent if, and only if, the transaction commits. It is also known as the transactional outbox pattern.

Updated · by the ChartCraft team · one of the patterns and models explained with a diagram

What does the outbox pattern look like as a diagram?

The diagram has one service, one database and one broker. The service writes to two tables, its business table and the outbox table, inside a single transaction. A message relay moves rows from the outbox to the message broker, and consumers read from the broker.

Outbox pattern diagram: a service writes an order and an outbox row in one transaction, a message relay publishes the row to a brokerOrder service database (one local transaction)Order service (sender)ORDERS table (business data)OUTBOX table (messages to send)Message relayMessage brokerConsumer service(idempotent)1. insert order2. insert message3. read new rows4. publish5. deliver
Outbox pattern: steps 1 and 2 commit together in one local transaction; the relay publishes afterwards. Rendered by ChartCraft's layout engine.

Mermaid code for this diagram

flowchart TD
    subgraph db["Order service database (one local transaction)"]
        orders[("ORDERS table (business data)")]
        outbox[("OUTBOX table (messages to send)")]
    end
    sender["Order service (sender)"]
    relay["Message relay"]
    broker["Message broker"]
    consumer["Consumer service (idempotent)"]
    sender -->|"1. insert order"| orders
    sender -->|"2. insert message"| outbox
    outbox -->|"3. read new rows"| relay
    relay -->|"4. publish"| broker
    broker -->|"5. deliver"| consumer

Paste it into any Mermaid renderer, or check and export it with the Mermaid syntax checker and Mermaid to PNG.

What are the parts of the outbox pattern?

The pattern has four participants: the sender, its database, the message outbox inside that database, and the message relay. The broker and the consumers are the systems the messages are for.

Part of the diagramWhat it isWhat it does
Order service (sender)The service that sends the messageHandles a command, changes its business data and writes the message to the outbox in the same transaction. It never calls the broker directly.
ORDERS table (business data)The business tablesHold the entities the command creates or updates, here a new order.
OUTBOX table (messages to send)The message outboxIn a relational database, a table that stores the messages to be sent. In a NoSQL database, the outbox is a property of each record instead.
Message relayA separate processSends the messages stored in the outbox to the message broker, in the order the service wrote them.
Message brokerThe messaging systemReceives each message from the relay and delivers it to subscribers.
Consumer service (idempotent)A service that reacts to the messageMust be idempotent, because the relay can publish the same message more than once.

How does the outbox pattern work, step by step?

The service does two inserts in one transaction, and the relay does the publishing later. Nothing is sent to the broker inside the request itself.

  1. A command arrives, for example place order.
  2. The service opens a database transaction and inserts or updates its business rows.
  3. In the same transaction it inserts a row describing the message, such as OrderCreated, into the outbox table.
  4. The transaction commits. If it rolls back, the business change and the message disappear together.
  5. The message relay finds the new outbox row and publishes the message to the broker.
  6. Consumers receive the message and ignore it if they have already processed its ID.

How does the message relay read the outbox?

There are two ways to implement the relay: a polling publisher, which queries the outbox table for unpublished rows, and transaction log tailing, which reads the database's transaction log and publishes each message inserted into the outbox.

Polling publisherTransaction log tailing
How it worksPublishes messages by polling the outbox tableTails the transaction log, for example the MySQL binlog, the Postgres WAL or DynamoDB table streams
StrengthWorks with any SQL databaseNo two-phase commit and guaranteed to be accurate
WeaknessTricky to publish events in order; not supported by every NoSQL databaseNeeds a database-specific solution, and avoiding duplicate publishing is tricky

When should you use the outbox pattern?

Use it when one operation has to change the database and send a message, and the two must succeed or fail together. That is the usual situation for a service that takes part in a saga or publishes domain events.

Use it when

Skip it when

What are common mistakes with the outbox pattern?

Most mistakes put the message outside the transaction or assume each message arrives exactly once.

How do you draw an outbox pattern diagram?

Start with the sender and its database, draw the business table and the outbox table inside one transaction boundary, then add the relay, the broker and one consumer, and number the arrows in the order they happen.

Opens the studio with your text · or start from the Architecture diagram generator

What do real event-driven architecture diagrams look like?

Live diagrams from the public ChartCraft gallery, each generated from a plain-English description. Open one to see the prompt that produced it.

Questions

What is the outbox pattern?

The outbox pattern is a way to update a database and publish a message reliably without a distributed transaction. The service writes the message to an outbox table in the same local transaction as its data change, and a separate message relay publishes it to the broker afterwards.

What problem does the outbox pattern solve?

It solves the problem of atomically updating a database and sending a message. Without it, a service that writes to the database and then calls the broker can crash between the two steps, leaving saved data with no message, or a message with no saved data.

Is the transactional outbox the same as the outbox pattern?

Yes. Transactional outbox is the name the pattern has in the microservices.io pattern catalogue; outbox pattern is the common short form.

Does the outbox pattern guarantee exactly-once delivery?

No. The message relay can publish a message more than once, for example if it crashes after publishing but before recording that it did. Consumers must be idempotent, typically by tracking the IDs of messages they have already processed.

What is the difference between a polling publisher and transaction log tailing?

They are the two ways to build the message relay. A polling publisher queries the outbox table for new rows and works with any SQL database. Transaction log tailing reads the database's transaction log, such as the MySQL binlog or Postgres WAL, and needs a database-specific solution.

More patterns and models explained with a diagram

Describe the system in plain English and get the diagram.

Architecture diagram generator →