The Programmer's Guide to Apache Thrift
Randy Abernethy
  • MEAP began July 2013
  • Publication in Early 2016 (estimated)
  • ISBN 9781617291814
  • 425 pages (estimated)
  • printed in black & white

The Programmer's Guide to Apache Thrift provides comprehensive coverage of the Apache Thrift framework along with a developer's-eye view of modern distributed application architecture. Packed with complete code examples and pragmatic discussion, this book lays the best practices for multi-language distributed application development. You'll take a guided tour through transports, protocols, IDL and servers as you explore complete example programs in C++, Java and Python. You'll also learn how to work with platforms ranging from enterprise servers to mobile clients.

Table of Contents show full

Part 1 APACHE THRIFT OVERVIEW

1. Introduction to Apache Thrift

1.1. Polyglotism, the pleasure and the pain

1.2. The Apache Thrift Framework from 30,000 feet

1.2.1. Data Normalization

1.2.2. Service Implementation

1.3. Apache Thrift Features

1.3.1. Service Implementation

1.3.2. Modular Serialization

1.3.3. Performance

1.3.4. Reach

1.3.5. Interface Evolution

1.4. Building a Simple Service

1.4.1. Describing services with Apache Thrift IDL

1.4.2. Building a Python Server

1.4.3. Building a Python Client

1.4.4. Building a C++ Client

1.4.5. Building a Java Client

1.5. Apache Thrift’s role in Distributed Applications

1.5.1. Streaming

1.5.2. Messaging

1.5.3. Remote Procedure Calls

1.6. The cross language communications landscape

1.6.1. REST, SOAP and XML-RPC

1.6.2. Google Protocol Buffers

1.6.3. Apache Avro

1.7. What Apache Thrift doesn’t do

1.8. Summary

2. Apache Thrift Architecture

2.1. Transports

2.1.1. The Transport Interface

2.1.2. End Point Transports

2.1.3. Layered Transports

2.1.4. Server Transports

2.2. Protocols

2.3. Apache Thrift IDL

2.3.1. User Defined Types & Serialization

2.3.2. RPC Services

2.4. Servers

2.5. Security

2.6. Summary

3. Building, Testing, and Debugging

3.1. Installing the Apache Thrift IDL Compiler

3.1.1. Platform Installers

3.1.2. VMs and Containers

3.1.3. Building from Source

3.1.4. Internet Binaries

3.2. The Apache Thrift Source Tree

3.3. Apache Thrift Tests

3.4. Debugging RPC Services

3.4.1. Examining Packets on the Wire

3.4.2. Unbuffered interfaces

3.4.3. Interface misalignment

3.4.4. I/O stack misalignment

3.4.5. Instrumenting Code

3.4.6. Additional Techniques

3.5. Summary

Part 2 PROGRAMMING APACHE THRIFT

4. Moving Bytes with Transports

4.1. End Point Transports - Part 1: Memory & Disk

4.1.1. Programming with Memory Transports

4.1.2. Programming with File Transports

4.2. The Transport Interface

4.2.1. Basic Transport Operations

4.2.2. Flushing Transport Buffers

4.2.3. Atomic Message Reads

4.2.4. Packaging Incremental Reads and Writes

4.2.5. Borrow/Consume

4.2.6. The Transport Use Pattern

4.3. End Point Transports - Part 2: Networks

4.3.1. Network Programming with TSocket

4.4. Server Transports

4.4.1. Programming Network Servers with Server Transports

4.4.2. The Server Transport Interface

4.5. Layered Transports

4.5.1. Message Framing

4.6. Summary

5. Serializing Data with Protocols

5.1. Basic Serialization with the Binary Protocol

5.1.1. Using the C++ TBinaryProtocol

5.1.2. Using the Java TBinaryProtocol

5.1.3. Using the Python TBinaryProtocol

5.1.4. Take Away

5.2. The TProtocol Interface

5.2.1. Apache Thrift Serialization

5.2.2. C++ TProtocol

5.2.3. Java TProtocol

5.2.4. Python TProtocolBase

5.3. Serializing Objects

5.3.1. Struct Serialization

5.3.2. Struct Deserialization

5.3.3. Struct Evolution

5.4. TCompactProtocol

5.5. TJSONProtocol

5.6. Selecting a Protocol

5.7. Summary

6. Apache Thrift IDL

6.1. Interfaces

6.2. Apache Thrift IDL

6.2.1. IDL File Names

6.2.2. Element Names

6.2.3. Keywords

6.3. The IDL Compiler

6.3.1. Compilation Phases and Error Messages

6.3.2. Command Line Switches

6.4. Comments and Documentation

6.5. Namespaces

6.6. Built-in Types

6.6.1. Base Types

6.6.2. Container Types

6.6.3. Literals

6.7. Constants

6.7.1. C++ Interface Constant Implementation

6.7.2. Java Interface Constant Implementation

6.7.3. Python Interface Constant Implementation

6.8. Typedefs

6.9. Enum

6.10. Structures, Unions, Exceptions and Argument-Lists

6.10.1. Structs

6.10.2. Fields

6.10.3. Exceptions

6.10.4. Unions

6.11. Services

6.11.1. Including External Files

6.11.2. Annotations

6.11.3. Summary

7. User Defined Types

7.1. A Simple User Defined Type Example

7.2. Type Design

7.2.1. Namespaces

7.2.2. Constants

7.2.3. Structs

7.2.4. Base Types

7.2.5. Typedefs

7.2.6. Field Ids and Retiring Fields

7.2.7. Enums

7.2.8. Collections

7.2.9. Unions

7.2.10. Requiredness and Optional Fields

7.3. Serializing Objects to Disk

7.4. Under the Type Serialization Hood

7.4.1. Serializing with write()

7.4.2. Deserializing with read()

7.5. Type Evolution

7.5.1. Renaming Fields

7.5.2. Adding Fields

7.5.3. Deleting Fields

7.5.4. Changing a Field’s Type

7.5.5. Changing a Field’s Requiredness

7.5.6. Changing a Field’s Default Value

7.6. Using Zlib Compression

7.6.1. Using Zlib with C++

7.6.2. Using Zlib with Python

7.7. Summary

8. Implementing Services

8.1. Declaring IDL Services

8.1.1. Parameter Identifiers

8.1.2. Parameter Requiredness

8.1.3. Default Parameter Values

8.1.4. Function and Parameter Types

8.2. Building a Simple Service

8.2.1. Interfaces

8.2.2. Coding Service Handlers and Test Harnesses

8.2.3. Coding RPC Servers

8.2.4. Coding RPC Clients

8.3. Service Interface Evolution

8.3.1. Adding Features to a Service

8.4. RPC Services In Depth

8.4.1. Under the Hood

8.4.2. Oneway Functions

8.4.3. Service Inheritance

8.4.4. Asynchronous Clients

8.5. Summary

9. Handling Exceptions

9.1. Apache Thrift Exceptions

9.2. TTransportException

9.2.1. C++ Exception Processing

9.2.2. Java Exception Processing

9.2.3. Python Exception Processing

9.2.4. Error Processing without Exceptions

9.3. TProtocolException

9.4. User Defined Exceptions

9.4.1. User Defined Exception IDL Example

9.4.2. C++ User Defined Exception Client

9.4.3. C++ User Defined Exception Server

9.4.4. Java User Defined Exception Client

9.4.5. Python User Defined Exception Client

9.5. Summary

10. Servers

10.1. Building a Simple Server from Scratch

10.2. Using Multithreaded Servers

10.3. Server Concurrency Models

10.3.1. Connection Based Processing

10.3.2. Task Based Processing

10.3.3. Multithreading vs. Multiprocessing

10.3.4. Server Summary by Language

10.4. Using Factories

10.4.1. Building I/O Stacks with Factories

10.4.2. Processor and Handler Factories

10.4.3. In/Out Factories

10.4.4. Building Servers with Custom Factories and Transports

10.5. Server Interfaces and Event Processing

10.5.1. TServer

10.5.2. TServerEventHandler

10.5.3. Building a C++ Thread Pool Server with Server Events

10.6. Servers and Services

10.6.1. Building Multiservice Servers

10.6.2. Building a Multiplexed Java Threaded Selector Server

10.7. Summary

Part 3 POLYGLOT APPLICATION DEVELOPMENT

11. Building Clients and Servers with C++

11.1. Setting up Apache Thrift for C++ Development

11.1.1. Apache Thrift C++ Versions and Boost

11.1.2. Building Apache Thrift C++ Libraries

11.1.3. Building Apache Thrift C++ Libraries on Windows

11.2. A Simple Client and Server

11.2.1. The Hello IDL

11.2.2. Building a Simple C++ Client

11.2.3. Creating a Simple RPC Server

11.3. C++ Transports, Protocols and Servers

11.3.1. C++ Transports

11.3.2. C++ Protocols

11.3.3. Runtime versus Compile Time Polymorphism

11.3.4. C++ Servers

11.4. The C++ TNonBlockingServer

11.5. Summary

12. Building Clients and Servers with Java

12.1. Setting up Apache Thrift for Java Development

12.2. A Simple Client and Server

12.2.1. The Hello IDL

12.2.2. Building a Simple Java Client

12.2.3. Creating a Simple RPC Server

12.2.4. Building with Ant

12.2.5. Building with Maven

12.3. Using Apache Thrift in other JVM languages

12.4. Java Transports, Protocols and Servers

12.4.1. Java Transports

12.4.2. Java Protocols

12.4.3. Java Servers

12.5. Asynchronous Java RPC

12.6. Summary

13. Building C# Clients and Servers with .Net and Windows

13.1. Setting up Apache Thrift on Windows

13.2. A Simple Client and Server

13.2.1. Creating a Visual Studio RPC Solution

13.2.2. Creating the Interface Library

13.2.3. Creating the RPC Server

13.2.4. Creating the RPC Client

13.2.5. Testing the RPC Application

13.3. C# Transports, Protocols and Servers

13.3.1. C# Transports

13.3.2. C# Protocols

13.3.3. C# Servers

13.4. Long Polling with Named Pipes

13.4.1. A Long Polling Interface

13.4.2. Installing Apache Thrift Support through NuGet

13.4.3. Creating a Named Pipe Server

13.4.4. Building the Long Polling Server

13.4.5. Building a Named Pipe Client

13.5. Summary

14. Building Node.js Clients and Servers

14.1. A Simple Client and Server

14.1.1. Generating the Client/Server Stubs

14.1.2. Creating a Node.js Server

14.1.3. Creating a Node.js Client

14.2. Q

14.3. Node.js Servers

14.4. Multiplexed Services

14.5. Node HTTP Clients

14.6. Apache Thrift IDL and Node.js

14.7. Summary

15. Apache Thrift and the Web

15.1. Apache Thrift JavaScript Quick Start

15.2. A Simple Client and Server

15.2.1. Installing Apache Thrift for JavaScript

15.2.2. The Hello World IDL

15.2.3. The Hello World Node.js Server

15.2.4. The Hello World Web Client

15.2.5. Running the Hello World Example

15.2.6. Debugging

15.3. Browser Asynchronous Client Calls

15.4. RPC Error Handling

15.5. Browser RPC and jQuery

15.6. Apache Thrift and Web Security

15.6.1. Cross-Origin Resource Sharing (CORS)

15.6.2. Content Security Policy (CSP)

15.6.3. X-Frame-Options

15.6.4. Transport Security

15.7. Using the Web Socket Transport

15.8. Summary

16. Scripting Apache Thrift

16.1. Apache Thrift and Ruby

16.1.1. A Ruby Server

16.1.2. A Ruby Client

16.1.3. Ruby Features

16.2. Apache Thrift and PHP

16.2.1. A PHP Program

16.2.2. A PHP Apache Thrift Client

16.2.3. PHP Features

16.3. Apache Thrift and Perl

16.4. Apache Thrift Perl Clients

16.5. Apache Thrift Perl Servers

16.5.1. Apache Thrift Perl Features

16.6. Apache Thrift and Haxe

16.6.1. An Apache Thrift Haxe Client

16.6.2. Haxe Features

16.7. Summary

17. Mobile Clients

18. The Big Picture

About the Technology

Any distributed application includes individual components, often written in different languages and hosted in multiple locations, which must communicate quickly and efficiently. Apache Thrift is a communication framework that enables cross-language remote procedure calls and serialization. Apache Thrift supports embedded, mobile, web, and server environments and a host of languages ranging from JavaScript to C++. It's perfect for back end services and embedded systems where size, scalability and performance are mission critical.

What's inside

  • Clear, concise coverage of all of the primary Apache Thrift features
  • Complete coverage of the Apache Thrift Interface Definition Language
  • Building and serializing complex user defined types
  • Working with plug in serialization protocols and data compression
  • Creating cross-language services
  • Tools and features to enable interface evolution

About the reader

This book assumes you're comfortable with a mainstream programming language like Java or C++ and the basics of service-oriented architecture. No experience with Apache Thrift is required.

About the author

Randy Abernethy is an active Apache Thrift contributor and can be found on the dev and user email lists. A serial entrepreneur, Randy founded Hollywood's first all hard disk recording studio in the early 90s, a direct market access institutional brokerage in the 2000s, and has recently focused on the development of proprietary automated trading systems using Apache Thrift for interoperability.


Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
MEAP combo $59.99 pBook + eBook
MEAP eBook $47.99 pdf + ePub + kindle

FREE domestic shipping on three or more pBooks