This chapter introduces Apache Thrift as a cross-language framework organized into five cooperating layers: transports, serialization protocols, user-defined type serialization, IDL-compiled RPC service stubs, and a reusable RPC server library. The lower three layers are sufficient when applications only need portable serialization, while adding stubs and servers enables full RPC. The design is conceptually object-oriented and centers on clear, T-prefixed abstractions that fit together to support storage, messaging, and cross-language services.
The transport layer abstracts device I/O and can target memory, files, or networks, with end point transports such as sockets and HTTP, and layered transports that add capabilities like framing or buffering; client and server must use compatible stacks. Server transports listen and accept connections, handing off a per-connection transport for I/O. Above transports, protocols handle type serialization for Thrift’s type system, dividing labor so transports move bytes while protocols encode data. Common choices include the fast Binary protocol, the size-efficient Compact protocol, and the human-readable JSON protocol.
Thrift’s IDL defines user types (structs) and services, and the compiler generates language-specific types with built-in read/write methods plus client proxies and server processors. An RPC call flows as the client stub serializes a message, sends it, and awaits a reply or exception; the processor deserializes, dispatches to a user-written handler, and returns results. Server libraries provide ready-made concurrency models to host services with minimal boilerplate. Security is kept external to the core, enabling environments to apply network isolation or integrate mechanisms like TLS or SASL—or custom layers—via the same composable transport model.
The Apache Thrift framework
Apache Thrift transports
Multiple I/O targets can be used interchangeably if they expose a common interface, such as TTransport.
Abstract transport interface
Interprocess communications using the TSocket end point transport
Layered transport stack
Server transport and I/O transports
Apache Thrift protocols
Serialization stack
IDL compilation
User-defined types
Apache Thrift RPC services
Thrift RPC call processing
Thrift servers
Summary
Key points from this chapter
Transports provide device independence for the rest of the Apache Thrift framework.
o End point transports perform byte I/O on physical or logical devices, such as networks, files, and memory.
o Layered transports add functionality to existing transports in a modular fashion, such as message framing and buffering.
o Any number of layered transports can be stacked on top of a single end point transport to create a transport stack.
Server transports aren’t true transports, rather they are factories, accepting client connections and manufacturing new transports for each connecting client.
Protocols are modular serialization engines. The primary Apache Thrift protocols are
o Binary: Simple and fast binary representations of data
o Compact: Trades CPU overhead for a smaller data footprint
o JSON: Trades speed and size for readability and broad interoperability
Apache Thrift IDL allows user-defined types and service interfaces to be defined.
The Apache Thrift IDL Compiler generates self-serializing representations of IDL user-defined types in various output languages.
The Apache Thrift IDL Compiler generates client and server stubs for IDL defined service interfaces in various output languages.
The Apache Thrift Server library allows IDL defined services to be deployed with minimal coding effort and a range of concurrency models.
FAQ
What are the five layers of the Apache Thrift framework?The layers are, from bottom to top: 1) Transport Library, 2) Serialization Protocol Library, 3) User-Defined Type Serialization (IDL-generated structs), 4) RPC Service Stubs (IDL-generated clients/processors), and 5) RPC Server Library. Many apps that only need cross-language serialization can use just the bottom three; the top two add RPC.What does the TTransport interface provide?TTransport abstracts byte-oriented I/O so upper layers don’t care about the device. Typical methods include open, close, isOpen, read, readAll, write, and flush. Because all transports share this interface, you can swap implementations (e.g., socket, file, memory, HTTP) at compile time or run time without changing protocol or application code.What’s the difference between end point transports and layered transports?End point transports sit at the bottom of the stack and talk to a device or protocol (e.g., TSocket for TCP/IP, TMemoryBuffer, file transports). Layered transports wrap another transport to add behavior such as framing (TFramedTransport), buffering (TBufferedTransport), or custom features like encryption. Layers are composable and device-independent.Do clients and servers need matching transport/protocol stacks?Yes. Client and server must use compatible transport and protocol configurations. For example, if the server uses TFramedTransport over TSocket, the client must do the same. Thrift doesn’t include runtime negotiation; agree on the stack out of band or implement your own discovery.What is a server transport and what methods does it expose?A server transport (TServerTransport) listens for connections and produces per-connection I/O transports. Core methods are listen (start accepting), accept (block until a client connects; returns a TTransport), close (stop), and interrupt (break out of a blocking accept). TServerSocket commonly manufactures TSockets for TCP/IP.What is a Thrift protocol and how do Binary, Compact, and JSON compare?A protocol defines how Thrift types are serialized over a transport. Binary is simple and fast with minimal CPU overhead. Compact reduces on-the-wire size using tighter encodings (more CPU, less I/O). JSON is human-readable and broadly interoperable but typically largest and most CPU intensive.How do the Thrift IDL and compiler enable cross-language development?You define structs and services in Thrift IDL once, then run the compiler to generate serialization code and RPC stubs for target languages (e.g., Java, Python, Go). A client in one language can call a server in another using the generated types and stubs.How are user-defined types (structs) serialized?The compiler generates a language-specific type for each IDL struct with read(protocol) and write(protocol) methods. These call TProtocol methods (like writeI16, writeString) to serialize fields. Thrift also generates internal structs for service method args and results, reusing the same read/write mechanism.How does a Thrift RPC call flow on the client side?The generated client stub method typically calls send_method(args) then recv_method(). send_ writes a message (writeMessageBegin with T_CALL, method name), serializes an args struct, and flushes the transport. recv_ reads the reply message, deserializes the result struct, and returns the value or raises an exception.What components handle RPC on the server side and how do you start a server?The processor (IDL-generated) deserializes requests, dispatches to your service handler (your implementation), and serializes results. To run a server, create a TServerTransport (e.g., TServerSocket), your handler, a Processor(handler), choose a TServer implementation (e.g., TSimpleServer, thread pool, async), and call serve(). Concurrency models vary by language.
pro $24.99 per month
access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
Programmer’s Guide to Apache Thrift ebook for free
