RabbitMQ in Depth
Gavin M. Roy
  • September 2017
  • ISBN 9781617291005
  • 264 pages
  • printed in black & white
ePub + Kindle available Oct 20, 2017

An excellent resource for beginners and experts alike...shows how to integrate RabbitMQ into a successful enterprise application.

Ian Dallas, Hewlett-Packard

RabbitMQ in Depth is a practical guide to building and maintaining message-based applications. This book provides detailed coverage of RabbitMQ with an emphasis on why it works the way it does. You'll find examples and detailed explanations based in real-world systems ranging from simple networked services to complex distributed designs. You'll also find the insights you need to make core architectural choices and develop procedures for effective operational management.

Table of Contents detailed table of contents

Part 1: RabbitMQ and Application Architecture

1. Foundational RabbitMQ

1.1. RabbitMQ’s features and benefits

1.1.1. RabbitMQ and Erlang

1.1.2. RabbitMQ and AMQP

1.2. Who is using RabbitMQ, and how?

1.3. The advantages of loosely coupled architectures

1.3.1. Decoupling your application

1.3.2. Decoupling database writes

1.3.3. Seamlessly adding new functionality

1.3.4. Replication of Data and Events

1.3.5. Multi-Master Federation of data and events

1.3.6. The Advanced Message Queuing Model

1.4. Summary

2. How to speak Rabbit: The AMQ protocol

2.1. AMQP as an RPC transport

2.1.1. Kicking off the conversation

2.1.2. Tuning in to the right channel

2.2. AMQP’s RPC Frame Structure

2.2.1. AMQP frame components

2.2.2. Types of frames

2.2.3. Marshaling messages into frames

2.2.4. The anatomy of a method frame

2.2.5. The content header frame

2.2.6. The body frame

2.3. Putting the protocol to use

2.3.1. Declaring an exchange

2.3.2. Declaring a queue

2.3.3. Binding a queue to an exchange

2.3.4. Publishing a Message to RabbitMQ

2.3.5. Consuming messages from RabbitMQ

2.4. Writing a message publisher in Python

2.5. Getting messages from RabbitMQ

2.6. Summary

3. An In-Depth Tour of Message Properties

3.1. Using properties properly

3.2. Creating an explicit message contract with content-type

3.3. Reducing message size with gzip and content-encoding

3.4. Referencing messages with message-id and correlation-id

3.5. Born-on dating: The timestamp property

3.6. Automatically expiring messages

3.7. Balancing speed with safety using delivery-mode

3.8. Validating message origin with app-id and user-id

3.8.1. app-id

3.8.2. user-id

3.9. Getting specific with the message type property

3.10. Using reply-to for dynamic workflows

3.11. Custom properties using the headers property

3.12. The priority property

3.13. A property you can’t use: cluster-id/reserved

3.14. Summary

4. Performance tradeoffs in publishing

4.1. Balancing delivery speed with guaranteed delivery

4.1.1. What to expect with no guarantees

4.1.2. RabbitMQ won’t accept non-routable messages with mandatory set

4.1.3. Publisher Confirms as a lightweight alternative to transactions

4.1.4. Using alternate-exchanges for unroutable messages

4.1.5. Batch processing with transactions

4.1.6. Surviving node failures with HA queues

4.1.7. HA queues with transactions

4.1.8. Persisting messages to disk via delivery-mode 2

4.2. When rabbits push back: TCP Backpressure

4.2.1. Checking the Client Status with rabbitpy

4.2.2. Using the Management API to detect backpressure

4.3. Summary

5. Don’t Get Messages; Consume Them

5.1. Basic.Get vs Basic.Consume

5.1.1. Basic.Get

5.1.2. Basic.Consume

5.2. Performance tuning consumers

5.2.1. Using No-Ack Mode for Faster Throughput

5.2.2. Controlling consumer prefetching via quality of service settings

5.2.3. Using Transactions with Consumers

5.3. Rejecting messages

5.3.1. Basic.Reject

5.3.2. Basic.Nack

5.3.3. Dead Letter Exchanges

5.4. Controlling queues

5.4.1. Temporary queues

5.4.2. Permanent queues

5.4.3. Arbitrary queue settings

5.5. Summary

6. Message Patterns via Exchange Routing

6.1. Simple message routing using the Direct exchange

6.1.1. Creating the application architecture

6.1.2. Creating the RPC Worker

6.1.3. Writing a simple RPC publisher

6.2. Broadcasting messages via the Fanout exchange

6.2.1. Modifying the facial detection consumer

6.2.2. Creating a simple image hashing consumer

6.3. Selectively routing messages with the Topic exchange

6.4. Selective routing with the Headers exchange

6.5. Exchange performance benchmarking

6.6. Going Meta: Exchange-to-exchange routing

6.7. Routing messages with the Consistent-Hashing exchange

6.8. Summary

Part 2: Managing RabbitMQ in the data center or the cloud

7. Scaling RabbitMQ with Clusters

7.1. About clusters

7.1.1. Clusters and the Management UI

7.1.2. Cluster node types

7.1.3. Clusters and queue behavior

7.2. Cluster setup

7.2.1. Virtual Machine setup

7.2.2. Adding nodes to the cluster

7.3. Summary

8. Cross-Cluster Message Distribution

8.1. Federating exchanges and queues

8.1.1. Federated Exchanges

8.1.2. Federated Queues

8.2. Creating the RabbitMQ Virtual Machines

8.2.1. Creating the first instance

8.2.2. Duplicating the EC2 instance

8.3. Connecting upstream

8.3.1. Defining federation upstreams

8.3.2. Defining a policy

8.3.3. Leveraging upstream sets

8.3.4. Bi-directional federated exchanges

8.3.5. Federation for cluster upgrades

8.4. Summary

Part 3: Integrations and Customization

9. Using Alternative Protocols

9.1. MQTT and RabbitMQ

9.1.1. The MQTT Protocol

9.1.2. Publishing via MQTT

9.1.3. MQTT Subscribers

9.1.4. MQTT Plugin Configuration

9.2. STOMP and RabbitMQ

9.2.1. The STOMP Protocol

9.2.2. Publishing Messages

9.2.3. Consuming Messages

9.2.4. Configuring the STOMP Plugin

9.2.5. Using STOMP in the web browser

9.3. Stateless Publishing via HTTP

9.3.1. How statelessd came to be

9.3.2. Using statelessd

9.3.3. Operational architecture

9.3.4. Publishing Messages via statelessd

9.4. Summary

10. Database Integrations

10.1. The pg_amqp PostgreSQL Extension

10.1.1. Installing the pg_amqp extension

10.1.2. Configuring the pg_amqp extension

10.1.3. Publishing a Message via pg_amqp

10.2. Listening to PostgreSQL Notifications

10.2.1. Installing the PostgreSQL LISTEN Exchange

10.2.2. Policy Based Configuration

10.2.3. Creating the Exchange

10.2.4. Creating and Binding a Test Queue

10.2.5. Publishing via NOTIFY

10.3. Storing Messages in InfluxDB

10.3.1. InfluxDB Installation and Setup

10.3.2. Installation of the InfluxDB Storage Exchange

10.3.3. Creating a Test Exchange

10.3.4. Testing the Exchange

10.4. Summary

Appendix A: Getting Set Up

A.1. Installing VirtualBox

A.2. Installing Vagrant

A.3. Setting up the Vagrant Virtual Machine

A.4. Confirming your installation

A.5. Summary

About the Technology

At the heart of most modern distributed applications is a queue that buffers, prioritizes, and routes message traffic. RabbitMQ is a high-performance message broker based on the Advanced Message Queueing Protocol. It?s battle tested, ultrafast, and powerful enough to handle anything you can throw at it. It requires a few simple setup steps, and you can instantly start using it to manage low-level service communication, application integration, and distributed system message routing.

What's inside

  • AMQP, the Advanced Message Queueing Protocol
  • Communicating via MQTT, Stomp, and HTTP
  • Valuable troubleshooting techniques
  • Database integration

About the reader

Written for programmers with a basic understanding of messaging-oriented systems.

About the author

Gavin M. Roy is an active, open source evangelist and advocate who has been working with internet and enterprise technologies since the mid-90s. Technical editor James Titcumb is a freelance developer, trainer, speaker, and active contributor to open source projects.

combo $59.99 pBook + eBook + liveBook
eBook $47.99 pdf + ePub + kindle + liveBook

FREE domestic shipping on three or more pBooks

The most comprehensive source for everything RabbitMQ. From terms to code to patterns, it's all here!

Andrew Meredith, Quantum Metric

A cheat sheet for getting started and troubleshooting the migration process to RabbitMQ.

Nadia Noori, La Salle University Barcelona

Filled with pragmatic advice and pearls of wisdom.

Milos Milivojevic, Mozzart Bet