RabbitMQ in Depth
Gavin M. Roy
  • MEAP began June 2013
  • Publication in August 2017 (estimated)
  • ISBN 9781617291005
  • 250 pages (estimated)
  • printed in black & white

RabbitMQ in Depth is a practical guide to building and maintaining message-based systems. This book covers detailed architectural and operational use of RabbitMQ with an emphasis on not just how it works but why it works the way it does. You'll find examples and detailed explanations of everything from low-level communication to integration with third-party systems. You'll also find the insights you need to make core architectural choices and develop procedures for effective operational management.

Table of Contents detailed table of contents

Part 1: RabbitMQ and Application Architecture

1. Foundational RabbitMQ

1.1. RabbitMQ’s features and benefits

1.1.1. RabbitMQ and Erlang

1.1.2. RabbitMQ and AMQP

1.2. Who is using RabbitMQ, and how?

1.3. The advantages of loosely coupled architectures

1.3.1. Decoupling your application

1.3.2. Decoupling database writes

1.3.3. Seamlessly adding new functionality

1.3.4. Replication of Data and Events

1.3.5. Multi-Master Federation of data and events

1.4. The Advanced Message Queuing Model

1.5. Summary

2. How to speak Rabbit: The AMQ protocol

2.1. AMQP as an RPC transport

2.1.1. Kicking off the conversation

2.1.2. Tuning in to the right channel

2.2. AMQP’s RPC Frame Structure

2.2.1. AMQP frame components

2.2.2. Types of frames

2.2.3. Marshaling messages into frames

2.2.4. The anatomy of a method frame

2.2.5. The content header frame

2.2.6. The body frame

2.3. Putting the protocol to use

2.3.1. Declaring an exchange

2.3.2. Declaring a queue

2.3.3. Binding a queue to an exchange

2.3.4. Publishing a Message to RabbitMQ

2.3.5. Consuming messages from RabbitMQ

2.4. Writing a message publisher in Python

2.5. Getting messages from RabbitMQ

2.6. Summary

3. An In-Depth Tour of Message Properties

3.1. Using properties properly

3.1.1. Creating an explicit message contract with content-type

3.1.2. Reducing message size with gzip and content-encoding

3.1.3. Referencing messages with message-id and correlation-id

3.1.4. Born-on dating: The timestamp property

3.1.5. Automatically expiring messages

3.1.6. Balancing speed with safety using delivery-mode

3.1.7. Validating message origin with app-id and user-id

3.1.8. Getting specific with the message type property

3.1.9. Using reply-to for dynamic workflows

3.1.10. Custom properties using the headers property

3.1.11. Avoiding the priority property

3.1.12. A property you can’t use: cluster-id/reserved

3.2. Summary

4. Performance tradeoffs in publishing

4.1. Balancing delivery speed with guaranteed delivery

4.1.1. What to expect with no guarantees

4.1.2. RabbitMQ won’t accept non-routable messages with mandatory set

4.1.3. Publisher Confirms as a lightweight alternative to transactions

4.1.4. Using alternate-exchanges for unroutable messages

4.1.5. Batch processing with transactions

4.1.6. Surviving node failures with HA queues

4.1.7. HA queues with transactions

4.1.8. Persisting messages to disk via delivery-mode 2

4.2. When rabbits push back: TCP Backpressure

4.2.1. Checking the Client Status with rabbitpy

4.2.2. Using the Management API to detect backpressure

4.3. Summary

5. Don’t Get Messages; Consume Them

5.1. Basic.Get vs Basic.Consume

5.1.1. Basic.Get

5.1.2. Basic.Consume

5.2. Performance tuning consumers

5.2.1. Using No-Ack Mode for Faster Throughput

5.2.2. Controlling consumer prefetching via quality of service settings

5.2.3. Using Transactions with Consumers

5.3. Rejecting messages

5.3.1. Basic.Reject

5.3.2. Basic.Nack

5.3.3. Dead Letter Exchanges

5.4. Controlling queues

5.4.1. Temporary queues

5.4.2. Permanent queues

5.4.3. Arbitrary queue settings

5.5. Summary

6. Message Patterns via Exchange Routing

6.1. Simple message routing using the Direct exchange

6.1.1. Creating the application architecture

6.1.2. Creating the RPC Worker

6.1.3. Writing a simple RPC publisher

6.2. Broadcasting messages via the Fanout exchange

6.2.1. Modifying the facial detection consumer

6.2.2. Creating a simple image hashing consumer

6.3. Selectively routing messages with the Topic exchange

6.4. Selective routing with the Headers exchange

6.5. Exchange performance benchmarking

6.6. Going Meta: Exchange-to-exchange routing

6.7. Routing messages with the Consistent-Hashing exchange

6.8. Summary

Part 2: Managing RabbitMQ in the data center or the cloud

7. Scaling RabbitMQ with Clusters

7.1. About clusters

7.1.1. Clusters and the Management UI

7.1.2. Cluster node types

7.1.3. Clusters and queue behavior

7.2. Cluster setup

7.2.1. Virtual Machine setup

7.2.2. Adding nodes to the cluster

7.3. Summary

8. Cross-Cluster Message Distribution

8.1. Federating exchanges and queues

8.1.1. Federated Exchanges

8.1.2. Federated Queues

8.2. Creating the RabbitMQ Virtual Machines

8.2.1. Creating the first instance

8.2.2. Duplicating the EC2 instance

8.3. Connecting upstream

8.3.1. Defining federation upstreams

8.3.2. Defining a policy

8.3.3. Leveraging upstream sets

8.3.4. Bi-directional federated exchanges

8.3.5. Federation for cluster upgrades

8.4. Summary

Part 3: Integrations and Customization

9. Using Alternative Protocols

9.1. MQTT and RabbitMQ

9.1.1. The MQTT Protocol

9.1.2. Publishing via MQTT

9.1.3. MQTT Subscribers

9.1.4. MQTT Plugin Configuration

9.2. STOMP and RabbitMQ

9.2.1. The STOMP Protocol

9.2.2. Publishing Messages

9.2.3. Consuming Messages

9.2.4. Configuring the STOMP Plugin

9.2.5. Using STOMP in the web browser

9.3. Stateless Publishing via HTTP

9.3.1. How statelessd came to be

9.3.2. Using statelessd

9.3.3. Operational architecture

9.3.4. Publishing Messages via statelessd

9.4. Summary

10. Database Integrations

10.1. The pg_amqp PostgreSQL Extension

10.1.1. Installing the pg_amqp extension

10.1.2. Configuring the pg_amqp extension

10.1.3. Publishing a Message via pg_amqp

10.2. Listening to PostgreSQL Notifications

10.2.1. Installing the PostgreSQL LISTEN Exchange

10.2.2. Policy Based Configuration

10.2.3. Creating the Exchange

10.2.4. Creating and Binding a Test Queue

10.2.5. Publishing via NOTIFY

10.3. Storing Messages in InfluxDB

10.3.1. InfluxDB Installation and Setup

10.3.2. Installation of the InfluxDB Storage Exchange

10.3.3. Creating a Test Exchange

10.3.4. Testing the Exchange

10.4. Summary

Appendix A: Getting Set Up

A.1. Installing VirtualBox

A.2. Installing Vagrant

A.3. Setting up the Vagrant Virtual Machine

A.4. Confirming your installation

A.5. Summary

About the Technology

Any large application needs an efficient way to handle the constant messages passing between components in the system. Billed as "messaging that just works," the RabbitMQ message broker initially appeals to developers because it's lightweight, easy to set up, and low maintenance. They stick with it, though, because it's powerful, fast, and up to nearly anything you can throw at it. This book takes you beyond the basics and explores the challenges of clustering and distributing messages across enterprise-level data-centers using RabbitMQ.

What's inside

  • Understanding the AMQP model
  • Communicating via MQTT, Stomp, and HTTP
  • Valuable troubleshooting techniques
  • Integrating with Java technologies like Hadoop and Esper
  • Database integrations with PostgreSQL and Riak

About the reader

Written for programmers with a basic understanding of messaging oriented systems and RabbitMQ.

About the author

Gavin M. Roy is an active open-source evangelist and advocate who has been working with Internet and Enterprise technologies since the mid-90's. As the CTO of MeetMe.com, Gavin implemented RabbitMQ as the core of MeetMe's event-based scalable application architecture. Gavin is a member of the Python Software Foundation and is the maintainer of pika, a pure-Python implementation of the AMQP protocol for RabbitMQ.

Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
MEAP combo $59.99 pBook + eBook + liveBook
MEAP eBook $47.99 pdf + ePub + kindle + liveBook

FREE domestic shipping on three or more pBooks