RabbitMQ

What is RabbitMQ?

RabbitMQ is an open-source message broker that speaks AMQP, the Advanced Message Queuing Protocol. Producers publish messages to it, consumers receive them, and the broker is responsible for holding each message until someone has acknowledged it.

The phrase that best describes its design is a smart broker with dumb consumers. RabbitMQ knows about exchanges, routing rules, acknowledgements, retries and dead letters. A consumer is a small program that reads a message, does one thing and says it is done. That division of labour is why RabbitMQ fits systems where routing and delivery guarantees matter more than raw throughput.

It was written in Erlang in 2007, which is why it is unusually good at handling many concurrent connections and why it has a reputation for stability. It runs a full protocol rather than a minimal queue API, and that protocol is the source of both its power and its learning curve.

The AMQP model

The single most important idea in RabbitMQ is that a producer never publishes to a queue. It publishes to an exchange, and the exchange decides which queues receive a copy.

producer -> exchange --binding--> queue -> consumer
                     \--binding--> queue -> consumer

Five pieces make up the model:

• A producer opens a channel and calls basic.publish with an exchange name, a routing key and a body.
• An exchange receives every published message and routes it. It does not store anything unless it is bound to a queue.
• A binding is a rule connecting an exchange to a queue. It can carry a routing key or a pattern, and one exchange can bind to many queues.
• A queue stores messages in order until a consumer takes them.
• A consumer subscribes to a queue, receives deliveries, and acknowledges them.

This indirection is the whole point. A producer that publishes order.created.eu does not know whether one service, five services or nobody is listening. New consumers are added by binding a new queue, without touching the producer.

All work happens on a channel, which is a lightweight virtual connection multiplexed over a single TCP connection. Channels are not thread-safe, so the common pattern is one channel per task or per consumer.

import amqp from "amqplib";

const conn = await amqp.connect(process.env.AMQP_URL!);
const ch = await conn.createChannel();

Exchange types and routing

An exchange’s type determines how it matches a routing key to its bindings. There are four, and each has a clear use.

A direct exchange routes to queues whose binding key equals the routing key exactly. Use it to send a message to one specific queue, or to a small set that all share a key.

await ch.assertExchange("logs", "direct", { durable: true });
await ch.bindQueue("logs.errors", "logs", "error");
await ch.publish("logs", "error", body);

A fanout exchange ignores the routing key entirely and copies the message to every bound queue. It is pub/sub in its purest form: one publish, many independent consumers.

await ch.assertExchange("events", "fanout", { durable: true });
await ch.bindQueue("search-indexer", "events", "");
await ch.bindQueue("email-notifier", "events", "");

A topic exchange matches the routing key against a pattern. Keys are dot-separated words. * matches exactly one word and # matches zero or more, which makes topic exchanges the most flexible of the four.

await ch.assertExchange("orders", "topic", { durable: true });

await ch.bindQueue("eu-orders", "orders", "order.*.eu");
await ch.bindQueue("all-orders", "orders", "order.#");
await ch.publish("orders", "order.created.eu", body);

A headers exchange ignores the routing key and matches on message header attributes instead. It is rarely the right choice, because topic exchanges are easier to read and reason about, but it is useful when routing depends on several independent attributes.

If you remember one rule, make it this: choose the exchange type by the question you are asking. “Which exact queue?” is direct. “Everyone?” is fanout. “Which family of events?” is topic.

Acknowledgements, nack and prefetch

RabbitMQ’s delivery guarantee is built on acknowledgement. When a consumer receives a message, the broker marks it unacked but keeps it. Only when the consumer calls ack is the message removed. If the connection drops before that, the broker redelivers.

await ch.consume("orders.created", async (msg) => {
  if (!msg) return;

  try {
    await handleOrder(JSON.parse(msg.content.toString()));
    ch.ack(msg);
  } catch (err) {
    ch.nack(msg, false, false); // requeue: false, dead-letter instead
  }
});

nack (or the older reject) takes a requeue flag. Requeueing puts the message back at the head of the queue, which is right for a transient failure and wrong for a poison message that will fail forever. Sending it to a dead-letter exchange instead is the usual answer.

Prefetch, set with basic.qos, caps how many unacknowledged messages a consumer holds at once. Without it, the broker pushes messages as fast as it can and a slow consumer buffers thousands of them in memory.

await ch.prefetch(20);

Set prefetch to a small multiple of your real concurrency. Too high and one consumer hogs the queue; too low and the consumer sits idle waiting for the next round trip.

Durability and delivery guarantees

A message survives a broker restart only if three things are true, and it is easy to get one wrong.

• The queue must be declared durable: true.
• The message must be published with persistent: true.
• The exchange should be durable: true as well.

await ch.assertExchange("orders", "topic", { durable: true });
await ch.assertQueue("orders.created", { durable: true });

ch.publish("orders", "order.created.eu", body, { persistent: true });

Durability is about surviving a restart, not about guaranteeing delivery. For that, enable publisher confirms. Without confirms, publish is fire-and-forget: if the broker dies before writing the message, the producer never knows.

const ch = await conn.createConfirmChannel();

ch.publish("orders", "order.created.eu", body, { persistent: true }, (err) => {
  if (err) console.error("broker did not confirm", err);
  else console.log("message is safely queued");
});

Even with confirms and durable queues, delivery is at-least-once. A consumer can crash after doing the work but before acking, and the broker will redeliver. Exactly-once across a network is effectively impossible, so RabbitMQ makes the trade explicit and asks you to make handlers idempotent.

Dead-letter exchanges and retry queues

A dead-letter exchange (DLX) is an ordinary exchange that receives messages which a queue rejects, expires or drops. It is the mechanism behind retry queues and poison-message handling.

A message is dead-lettered when one of these happens:

• The consumer nacks or rejects it with requeue: false.
• Its per-message or per-queue TTL expires.
• The queue exceeds its length limit and drops the oldest message.

You configure the DLX on the queue that holds the work, not on the consumer.

await ch.assertQueue("orders.created", {
  durable: true,
  deadLetterExchange: "orders.dlx",
  deadLetterRoutingKey: "failed",
});

The classic delayed retry pattern uses a second queue with a TTL and its own DLX pointing back at the work exchange. A failed message is dead-lettered into the retry queue, sits there for the TTL, expires, and is dead-lettered back to be processed again. That produces a retry with a delay and no timer in your code.

await ch.assertExchange("orders.dlx", "direct", { durable: true });

await ch.assertQueue("orders.retry", {
  durable: true,
  messageTtl: 30_000,
  deadLetterExchange: "orders",
  deadLetterRoutingKey: "order.created.retry",
});

await ch.bindQueue("orders.retry", "orders.dlx", "retry");

A message that keeps failing will loop. Track a retry count in the message headers and, after a limit, route it to a permanent failed queue that nobody consumes automatically. Treat that queue as an operational surface: alert when it grows and build a replay path.

Message TTL and queue limits

Time-to-live controls how long a message may wait. Set it per queue, per message, or both.

await ch.assertQueue("verification", {
  durable: true,
  messageTtl: 600_000,          // 10 minutes for every message
  maxLength: 10_000,            // keep at most 10k messages
  overflow: "reject-publish",   // backpressure instead of dropping
});

Per-message TTL is set when publishing and is often used for values that differ by message.

ch.publish("orders", key, body, {
  expiration: "30000", // milliseconds, as a string
});

Queue length limits are backpressure. maxLength bounds memory, and overflow decides what happens when the limit is reached: drop-head silently discards the oldest message, while reject-publish refuses new publishes so the producer feels the pressure. For a queue that must not lose work, reject-publish is the safer default and an alerting signal.

Work queues vs pub/sub

The same broker covers two very different shapes of communication, and mixing them up causes bugs.

A work queue distributes each message to exactly one consumer. Many workers compete on the same queue, and the broker round-robins deliveries. Add a prefetch and acknowledgements and you have a scalable, fault-tolerant task queue.

await ch.assertQueue("jobs.thumbnails", { durable: true });
await ch.prefetch(5);

Pub/sub delivers each message to every interested consumer. Each subscriber gets its own queue bound to the same exchange, so a slow or offline subscriber never steals a message from the others.

await ch.assertExchange("events", "fanout", { durable: true });

await ch.assertQueue("events.billing", { durable: true });
await ch.bindQueue("events.billing", "events", "");

await ch.assertQueue("events.analytics", { durable: true });
await ch.bindQueue("events.analytics", "events", "");

The rule of thumb: one queue shared by many consumers is a work queue; one queue per consumer is pub/sub. A topic exchange lets you have both at once, with some consumers sharing a queue and others owning their own.

The RPC pattern

RabbitMQ can also do request/response. The client publishes a request with a replyTo queue and a correlationId, and the server publishes the response to that queue with the same id.

import { randomUUID } from "node:crypto";

const { queue } = await ch.assertQueue("", { exclusive: true });
const correlationId = randomUUID();

ch.consume(queue, (msg) => {
  if (msg?.properties.correlationId === correlationId) {
    console.log("reply", JSON.parse(msg.content.toString()));
  }
}, { noAck: true });

ch.publish("rpc.inventory", "check", Buffer.from("{}"), {
  replyTo: queue,
  correlationId,
});

The assertQueue("") creates a temporary, exclusive, auto-deleting queue just for this client. RPC over a queue is useful when the caller genuinely needs an answer, but it reintroduces synchronous coupling and a timeout problem. For most systems, an event plus a callback event is easier to operate than RPC, and a plain HTTP call is easier still when the dependency is available.

Clustering and quorum queues

A single node is a single point of failure, so production RabbitMQ runs as a cluster. Queues can be replicated across nodes, and clients reconnect to another node when one fails.

The modern replication strategy is the quorum queue, built on the Raft consensus algorithm. A quorum queue has a leader and followers, and a write is confirmed only once a majority has it. That makes it safe under the network partitions that caused classic mirrored queues to lose data, and it is the default choice for durable queues today.

await ch.assertQueue("orders.created", {
  durable: true,
  arguments: { "x-queue-type": "quorum" },
});

Quorum queues prefer a small, odd number of replicas, typically three or five. They are heavier than classic queues, so use them for data you cannot lose and keep transient queues classic. For multi-region topologies, federation and the shovel plugin move messages between independent brokers rather than stretching one cluster across a slow link.

The management UI and monitoring

Every RabbitMQ node ships with a management plugin that serves a web UI and an HTTP API. It shows exchanges, queues, bindings, connections and channels, and it lets you publish a test message or replay one from a queue.

rabbitmq-plugins enable rabbitmq_management

curl -u guest:guest http://localhost:15672/api/queues/%2F/orders.created

Four numbers matter most on a dashboard.

• Queue depth — messages ready. A rising depth means consumers cannot keep up.
• Unacked count — messages delivered but not acknowledged. A number that only grows means consumers are stuck.
• Publish and deliver rates — the shape of the traffic, and whether consumers keep pace.
• Redelivery rate — a rising redelivery count points at crashes or repeated failures.

RabbitMQ also emits Prometheus metrics, so queue depth and consumer utilisation belong on the same dashboard as the rest of your services. Alert on depth and oldest-message age, not on individual failures, which are expected.

Connections, channels and recovery

A connection is expensive; a channel is cheap. Open one connection per process, then create a channel per producer, per consumer or per unit of work. A channel is not thread-safe, so sharing one across concurrent handlers causes interleaved frames and confusing errors.

import amqp from "amqplib";

let conn: amqp.Connection;
let ch: amqp.Channel;

async function connect() {
  conn = await amqp.connect(process.env.AMQP_URL!);

  conn.on("error", (err) => console.error("connection error", err));
  conn.on("close", () => setTimeout(connect, 5_000));

  ch = await conn.createChannel();
  await ch.assertExchange("orders", "topic", { durable: true });
}

Handle reconnection deliberately. amqplib does not reconnect for you, so listen for close and rebuild the connection, channel and every consumer. Because a dropped connection leaves in-flight messages unacked, the broker redelivers them, which is the behaviour you want. This is another place where at-least-once delivery shows up: a reconnect can replay work, so handlers must be idempotent.

Message properties and priorities

Every published message can carry properties alongside its body. They travel with the message and are visible to consumers, which makes them a lightweight place for metadata.

ch.publish("orders", "order.created.eu", body, {
  persistent: true,
  contentType: "application/json",
  messageId: order.id,
  correlationId: traceId,
  timestamp: Math.floor(Date.now() / 1000),
  type: "order.created",
  headers: { "x-retry-count": 0, "x-source": "checkout" },
});

contentType and messageId help consumers and tooling; correlationId ties a message to a trace; headers are where you keep application metadata such as a retry count. Do not put large data in headers, because the broker must index and display them.

RabbitMQ also supports priority queues, where higher-priority messages are delivered first. Declare the queue with a maximum priority and publish with a priority value.

await ch.assertQueue("jobs", {
  durable: true,
  maxPriority: 10,
});

ch.publish("", "jobs", body, { priority: 9 });

Priorities only reorder messages that are already waiting. If consumers keep the queue empty, priority does nothing, and a high-priority message that arrives after a low one still waits behind it. Use priority for a genuine business distinction, not as a general scheduling mechanism.

Choosing RabbitMQ vs Kafka

The two are often compared, but they solve different problems.

RabbitMQ is a smart broker for routing and work queues. Messages are tasks that are removed once acknowledged. Exchanges route by pattern, acknowledgements are per message, and dead-lettering is built in. It shines when a message must reach a specific set of queues, or when a unit of work must be retried and parked on failure.

Kafka is a durable log. Messages are appended to a partitioned, ordered log and retained for a configured time regardless of who reads them. Many consumer groups can read the same topic independently, and any consumer can rewind to an earlier offset. It shines at very high throughput and at event replay.

A useful heuristic: if you would be upset that a message was consumed and deleted, you want a log. If you care that a task was routed correctly and completed, you want a broker. Many systems use both — Kafka for the event stream, RabbitMQ for the work.

Best practices

• Declare exchanges, queues and bindings idempotently at startup, and use durable: true for anything you cannot lose.
• Publish persistent messages to durable queues, and use publisher confirms for work that must not vanish.
• Always acknowledge manually, after the side effect succeeds, never before.
• Set a prefetch limit so one consumer cannot buffer the whole queue.
• Give every durable queue a dead-letter exchange, and build a replay path for it.
• Implement delayed retries with a TTL retry queue rather than sleeping in the handler.
• Make consumers idempotent, because delivery is at-least-once.
• Bound queue length and choose reject-publish when dropping messages is unacceptable.
• Use quorum queues for durable data and classic queues for transient data.
• Close channels and connections on SIGTERM, and stop consuming before draining.
• Watch ready, unacked, redelivery rate and oldest-message age.

Common mistakes

• Publishing to a queue instead of an exchange, then wondering why bindings do nothing.
• Auto-acking (noAck: true) and losing messages whenever a handler crashes.
• Forgetting persistent: true, then losing messages on a broker restart.
• Declaring a queue as durable but publishing non-persistent messages.
• Setting prefetch too high and letting one consumer starve the rest.
• Requeueing a poison message forever with nack(msg, false, true).
• Building a dead-letter exchange with no consumer and never looking at it.
• Using a fanout exchange where a topic exchange was needed, or the reverse.
• Running a single broker in production and calling it highly available.
• Mixing work-queue and pub/sub semantics on the same queue.
• Blocking the event loop in a consumer, which stops heartbeats and triggers a false disconnect.

Where to go next

If you want the same reliable-work idea with less infrastructure, the Redis Queues guide covers BullMQ on top of Redis. For the broader discipline of moving work out of the request path, read Batch Processing. When your events are a durable log that many consumers replay rather than tasks that are routed and removed, the Kafka guide is the natural next step, and Node.js basics covers the runtime every amqplib consumer runs on.