Apache Pulsar in Action
David Kjerrumgaard
  • MEAP began July 2019
  • Publication in Spring 2021 (estimated)
  • ISBN 9781617296888
  • 300 pages (estimated)
  • printed in black & white

Yes! Yes! Yes! This book is fantastic!

Kent Spillner

Distributed applications demand reliable, high-performance messaging. The Apache Pulsar server-to-server messaging system provides a secure, stable platform without the need for a stream processing engine like Spark. Contributed by Yahoo to the Apache Foundation, Pulsar is mature and battle-tested, handling millions of messages per second for over three years at Yahoo. Apache Pulsar in Action is a comprehensive and practical guide to building high-traffic applications with Pulsar, delivering extreme levels of speed and durability.

About the Technology

Pulsar is a streaming messaging system designed for high performance server-to-server messaging. Built and tested under intense conditions at Yahoo, Pulsar has been proven in production and can handle millions of messages per second. Now free and open-source, Pulsar’s unique architecture helps solve some of the challenges of modern development. Pulsar avoids latency in streaming data transmission, making it a powerful tool for IoT Edge analytics. Its unified messaging model improves the performance of microservices architecture, and its tiered storage capabilities allow for larger volumes of data to be handled without fear of data loss. Pulsar’s flexible API interface works with Java, C++, Python, and Go, making it easy to incorporate Pulsar into your stack.

About the book

Apache Pulsar in Action is a hands-on guide to building scalable streaming messaging systems for distributed applications and microservices systems. You’ll start with Pulsar’s fundamentals, each illustrated by real-world examples, as you get to grips with Pulsar’s unique architecture. Pulsar contributor David Kjerrumgaard teaches the skills you need to deploy a Pulsar server, ingest data from third-party systems, and deploy lightweight computing logic with simple functions. You’ll learn to employ Pulsar’s seamless scalability through relatable case studies, including an IOT analytics application that can be deployed within a resource constrained environment and a microservices application based on Pulsar functions. At the end of this practical book, you’ll be ready to fully take advantage of Pulsar to create high-traffic message-driven applications.

Table of Contents detailed table of contents

Part 1: Getting Started with Apache Pulsar

1 Introduction to Apache Pulsar

1.1 Enterprise Messaging Systems

1.1.1 Key Capabilities

1.2 Message Consumption Patterns

1.2.1 Message Queuing

1.2.2 Publish-Subscribe Messaging

1.3 The Evolution of Messaging Systems

1.3.1 Generic Messaging System

1.3.2 Message Oriented Middleware

1.3.3 Enterprise Message Bus

1.3.4 Distributed Messaging Systems

1.4 Comparison to Apache Kafka

1.4.1 Multi-Layered Architecture

1.4.2 Message Consumption

1.4.3 Data Durability

1.4.4 Message Acknowledgement

1.4.5 Message Retention

1.5 Why Do I Need Pulsar?

1.5.1 Guaranteed Message Delivery

1.5.2 Infinite Scalability

1.5.3 Resilient to Failure

1.5.4 Support for Millions of Topics

1.5.5 Geo-Replication and Active Failover

1.6 Real World Use Cases

1.6.1 Unified Messaging System

1.6.2 Microservices Platform

1.6.3 Connected Car

1.6.4 Fraud Detection

1.7 Additional Resources

1.8 Summary

2 Pulsar Concepts and Architecture

2.1 Pulsar’s Logical Architecture

2.1.1 Tenants, Namespaces, and Topics

2.1.2 Addressing Topics in Pulsar

2.1.3 Producers, Consumers, and Subscriptions

2.1.4 Subscription Types

2.2 Pulsar’s Physical Architecture

2.2.1 Pulsar’s Layered Architecture

2.2.2 Stateless Serving Layer

2.2.3 Stream Storage Layer

2.2.4 Metadata Storage

2.3 Message Retention and Expiration

2.3.1 Data Retention

2.3.2 Backlog Quotas

2.3.3 Message Expiration

2.3.4 Message Backlog vs. Message Expiration

2.4 Tiered Storage

2.5 Summary

3 Interacting with Pulsar

3.1 Getting Started with Pulsar

3.1.1 Administering Pulsar

3.1.2 Pulsar Client

3.2 Pulsar Java Client

3.2.1 Pulsar Client Configuration

3.2.2 Pulsar Producers

3.2.3 Pulsar Consumers

3.2.4 Pulsar Readers

3.3 Advanced Administration

3.3.1 Persistent Topic Metrics

3.3.2 Message Inspection

3.4 Summary

Part 2: Apache Pulsar Development Essentials

4 Pulsar Functions

4.1 Stream Native Processing

4.1.1 Traditional Batching

4.1.2 Micro-Batching

4.1.3 Stream Processing

4.2 What are Pulsar Functions?

4.2.1 Programming Model

4.3 Developing Pulsar Functions

4.3.1 Native Functions

4.3.2 The Pulsar SDK

4.3.3 Stateful Functions

4.4 Developing Pulsar Functions

4.4.1 Unit Testing

4.4.2 Integration Testing

4.5 Deploying Pulsar Functions

4.5.1 Generating a Deployment Artifact

4.5.2 Configuration and Deployment

4.5.3 The Function Deployment Lifecycle

4.5.4 Deployment Modes

4.5.5 Pulsar Function Data Flow

4.6 Summary

5 Pulsar IO Connectors

5.1 What are Pulsar IO Connectors?

5.1.1 Sources and Sinks

5.1.2 Programming Model

5.1.3 Pulsar’s Built-In Connectors

5.1.4 Using the Built-In Connectors

5.2 Developing Pulsar IO Connectors

5.2.1 Developing a Source Connector

5.2.2 Developing a Sink Connector

5.2.3 Packaging and Deploying Pulsar IO Connectors

5.3 Deploying Pulsar IO Connectors

5.4 Administering Pulsar IO Connectors

5.4.1 Creating and Deleting Connectors

5.4.2 Listing Connectors

5.4.3 Monitoring Connectors

5.5 Summary

6 Pulsar Security

6.1 Transport Encryption

6.2 Authentication

6.2.1 TLS Authentication

6.2.2 JSON Web Token Authentication

6.3 Authorization

6.3.1 Roles

6.3.2 An Example Scenario

6.4 Message Encryption

6.5 Summary

7 Schema Registry

7.1 Microservice Communication

7.1.1 Microservice APIs

7.1.2 The Need for a Schema Registry

7.2 The Pulsar Schema Registry

7.2.1 Architecture

7.2.2 Schema Versioning

7.2.3 Schema Compatibility

7.2.4 Schema Compatibility Check Strategies

7.3 Using the Schema Registry

7.3.1 Modelling the Food Order Event in Avro

7.3.2 Producing Food Order Events

7.3.3 Consuming the Food Order Events

7.3.4 Complete Example

7.4 Evolving the Schema

7.5 Summary

Part 3: Hands-On Application Development with Apache Pulsar

8 Pulsar Function Patterns

8.1 Data Pipelines

8.1.1 Procedural Programming

8.1.2 DataFlow Programming

8.2 Message Routing Patterns

8.2.1 Splitter

8.2.2 Dynamic Router

8.2.3 Content Based Router

8.3 Message Transformation Patterns

8.3.1 Message Translator

8.3.2 Content Enricher

8.3.3 Content Filter

8.4 Summary

9 Resiliency Patterns

9.1 Pulsar Function Resiliency

9.1.1 Adverse Events

9.1.2 Fault Detection

9.2 Resiliency Design Patterns

9.2.1 Retry Pattern

9.2.2 Circuit Breaker

9.2.3 Rate Limiter

9.2.4 Time Limiter

9.2.5 Cache

9.2.6 Fallback Pattern

9.2.7 Credential Refresh Pattern

9.3 Multiple Layers of Resiliency

9.4 Summary

10 Data Access Patterns

11 Machine Learning in Pulsar

12 IOT Edge Analytics


Appendix A: Running Pulsar in Containerized Environment

What's inside

  • Publish from Apache Pulsar into third-party data repositories and platforms
  • Design and develop Apache Pulsar functions
  • Perform interactive SQL queries against data stored in Apache Pulsar
  • Examples of Pulsar-based microservices that you can download and try yourself

About the reader

Written for experienced Java developers. No prior knowledge of Pulsar is needed.

About the author

David Kjerrumgaard is the Director of Solution Architecture at Streamlio, and a contributor to the Apache Pulsar and Apache NiFi projects.

placing your order...

Don't refresh or navigate away from the page.
Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
print book $29.99 $49.99 pBook + eBook + liveBook
Additional shipping charges may apply
Apache Pulsar in Action (print book) added to cart
continue shopping
go to cart

eBook $24.99 $39.99 3 formats + liveBook
Apache Pulsar in Action (eBook) added to cart
continue shopping
go to cart

Prices displayed in rupees will be charged in USD when you check out.

FREE domestic shipping on three or more pBooks