Event-Driven Architecture with NATS and RabbitMQ

Event-Driven Architecture Fundamentals

Event-driven architecture (EDA) is a design pattern where services communicate by producing and consuming events. Instead of Service A calling Service B directly, Service A publishes an event ("user registered") and any interested service can react to it.

Benefits:

• Loose coupling: Services do not know about each other
• Scalability: Add consumers without modifying producers
• Resilience: If a consumer is down, messages queue up
• Audit trail: Events form a natural log of what happened

NATS: The Cloud-Native Messaging System

NATS is an incredibly lightweight messaging system. The server binary is 15MB, uses 10MB of RAM, and handles millions of messages per second.

# docker-compose.yml
services:
  nats:
    image: nats:2.10-alpine
    container_name: nats
    command: ["-js", "-sd", "/data", "-m", "8222"]
    ports:
      - "4222:4222"   # Client connections
      - "8222:8222"   # HTTP monitoring
    volumes:
      - nats-data:/data
    mem_limit: 128m

NATS Core: Simple Pub/Sub

import { connect, StringCodec, JSONCodec } from 'nats';

const nc = await connect({ servers: 'nats://nats:4222' });
const jc = JSONCodec();

// Publisher
nc.publish('user.registered', jc.encode({
  userId: '123',
  email: '[email protected]',
  name: 'Jane Doe',
  timestamp: Date.now(),
}));

// Subscriber (receives messages while connected)
const sub = nc.subscribe('user.registered');
for await (const msg of sub) {
  const data = jc.decode(msg.data);
  console.log('New user:', data.email);
  await sendWelcomeEmail(data);
}

// Wildcard subscriptions
nc.subscribe('user.>');        // user.registered, user.updated, user.deleted
nc.subscribe('orders.*.shipped'); // orders.123.shipped, orders.456.shipped

// Request-Reply (synchronous over async)
const response = await nc.request('auth.validate', jc.encode({ token: 'abc' }), {
  timeout: 5000,
});
const authResult = jc.decode(response.data);

NATS JetStream: Persistent Messaging

JetStream adds persistence, replay, and exactly-once delivery:

const js = nc.jetstream();
const jsm = await nc.jetstreamManager();

// Create a stream (persists messages)
await jsm.streams.add({
  name: 'EVENTS',
  subjects: ['events.>'],
  retention: 'limits',
  max_msgs: 1000000,
  max_age: 7 * 24 * 60 * 60 * 1e9, // 7 days
  storage: 'file',
  num_replicas: 1,
});

// Publish to stream
await js.publish('events.order.created', jc.encode({
  orderId: 'ord-789',
  items: [{ sku: 'WIDGET-1', qty: 2 }],
}));

// Durable consumer (survives restarts, tracks position)
await jsm.consumers.add('EVENTS', {
  durable_name: 'order-processor',
  filter_subject: 'events.order.>',
  ack_policy: 'explicit',
  deliver_policy: 'all', // Start from the beginning
  max_deliver: 3,        // Max delivery attempts
});

const consumer = await js.consumers.get('EVENTS', 'order-processor');
const messages = await consumer.consume({ max_messages: 10 });

for await (const msg of messages) {
  try {
    const event = jc.decode(msg.data);
    await processOrder(event);
    msg.ack();
  } catch (error) {
    msg.nak(5000); // Negative ack, retry after 5 seconds
  }
}

RabbitMQ: The Feature-Rich Broker

RabbitMQ provides advanced routing, dead letter queues, priority queues, and management UI out of the box.

# docker-compose.yml
services:
  rabbitmq:
    image: rabbitmq:3.13-management-alpine
    container_name: rabbitmq
    environment:
      RABBITMQ_DEFAULT_USER: admin
      RABBITMQ_DEFAULT_PASS: secret
    ports:
      - "5672:5672"    # AMQP
      - "15672:15672"  # Management UI
    volumes:
      - rabbitmq-data:/var/lib/rabbitmq
    mem_limit: 512m

RabbitMQ Patterns

Direct Exchange (point-to-point):

import pika
import json

connection = pika.BlockingConnection(pika.ConnectionParameters('rabbitmq'))
channel = connection.channel()

# Declare exchange and queue
channel.exchange_declare(exchange='orders', exchange_type='direct', durable=True)
channel.queue_declare(queue='order-processing', durable=True)
channel.queue_bind(queue='order-processing', exchange='orders', routing_key='created')

# Publish
channel.basic_publish(
    exchange='orders',
    routing_key='created',
    body=json.dumps({'orderId': '123', 'total': 49.99}),
    properties=pika.BasicProperties(
        delivery_mode=2,  # Persistent message
        content_type='application/json',
    ),
)

# Consume
def process_order(ch, method, properties, body):
    order = json.loads(body)
    print(f"Processing order {order['orderId']}")
    # Process...
    ch.basic_ack(delivery_tag=method.delivery_tag)

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue='order-processing', 
channel.start_consuming()

Topic Exchange (pattern matching):

# Publisher
channel.exchange_declare(exchange='events', exchange_type='topic', durable=True)

channel.basic_publish(exchange='events', routing_key='user.registered.us',
                     body=json.dumps(user_data))
channel.basic_publish(exchange='events', routing_key='order.shipped.eu',
                     body=json.dumps(order_data))

# Subscriber (US users only)
channel.queue_bind(queue='us-notifications', exchange='events',
                  routing_key='user.*.us')

# Subscriber (all shipments)
channel.queue_bind(queue='shipping-tracker', exchange='events',
                  routing_key='order.shipped.*')

Dead Letter Queue (handle failures):

# Main queue with DLQ
channel.queue_declare(
    queue='order-processing',
    durable=True,
    arguments={
        'x-dead-letter-exchange': 'dlx',
        'x-dead-letter-routing-key': 'failed-orders',
        'x-message-ttl': 300000,    # 5 minute TTL
        'x-max-length': 10000,      # Max queue size
    },
)

# Dead letter queue
channel.exchange_declare(exchange='dlx', exchange_type='direct')
channel.queue_declare(queue='failed-orders', durable=True)
channel.queue_bind(queue='failed-orders', exchange='dlx', routing_key='failed-orders')

NATS vs RabbitMQ Comparison

When to Choose Each

Choose NATS when: You need ultra-high throughput (millions/sec), want minimal resource usage, your routing needs are simple (subject-based), or you are building cloud-native microservices.

Choose RabbitMQ when: You need complex routing (topic exchanges, headers), require dead letter queues and priority queues, need protocol compatibility (AMQP, MQTT, STOMP), or your team is already familiar with AMQP.

At TechSaaS, we use n8n for our event-driven workflows instead of a raw message broker. For client architectures that need messaging infrastructure, we recommend NATS for its simplicity and performance (15MB binary, 10MB RAM, handles millions of messages). RabbitMQ is our recommendation only when clients need its advanced routing and management UI for teams that prefer visual queue management.