10 Servers
Apache Thrift servers are the runtime backbone that host IDL-defined services, coordinate transport and protocol stacks, and hide the complexity of high‑performance RPC across languages. This chapter frames servers as the final piece in the stack—after transports, protocols, and services—showing how they manage connections, concurrency, and cross‑thread communication. It surveys how different languages offer different server choices (from JavaScript’s evented model to Java/C++’s rich back‑end options), and sets out the two principal concurrency models: connection‑based (one thread per connection) and task‑based (evented/nonblocking with worker pools). It also positions servers in modern, cloud‑native deployments where lightweight processes can be scaled horizontally by orchestration platforms.
Through progressively richer examples, the chapter moves from a minimal single‑threaded C++ server to pooled, threaded Java servers and nonblocking designs. It explains how connection‑based servers (TSimpleServer, TThreadedServer, TThreadPoolServer) trade simplicity and request ordering for higher thread counts, while task‑based servers (TNonblockingServer, THsHaServer, TThreadedSelectorServer) use nonblocking I/O, framing (TFramedTransport), and worker pools to scale to large connection sets with fewer threads. It details platform I/O differences (Java NIO selectors; C++ libevent; Python’s GIL driving multiprocessing via TForkingServer/TProcessPoolServer) and gives pragmatic performance guidance: thread‑per‑connection can excel at modest scale; task‑based designs shine at very high concurrency. A major theme is configurability via factories: TServerTransport, TTransportFactory, and TProtocolFactory let you compose custom I/O stacks (e.g., framed + JSON), define per‑connection handlers with processor/handler factories, and even use distinct input/output stacks for asymmetric behaviors like output compression or write‑side logging.
On the operational side, the TServer interface (serve/stop) and TServerEventHandler (preServe, createContext, processContext, deleteContext) enable lifecycle hooks for connection tracking, per‑connection context, metering, and adaptive behavior. The chapter demonstrates dynamic pool management with C++ ThreadManager to grow/shrink worker threads in a TThreadPoolServer, and shows how to run servers on background threads for graceful shutdown. It concludes with service multiplexing, where TMultiplexedProcessor and TMultiplexedProtocol allow a single server/port to host many services, reducing ports, connections, and operational overhead. Taken together, these patterns let you choose the right server for your language and workload, assemble the I/O and handler model you need, instrument it with events, and—most importantly—validate choices under realistic load before production.
- The Apache Thrift framework server library
– Connection based single threaded server state diagram
– Connection based multithreaded server state diagram
– Threading model for connection based processing
- Threading model for task based processing using a pool of worker threads, any thread may handle requests from any client
- Threading model for task based processing using a single worker thread
- Threading model for task based processing using connection partitioning
- Threading model for task based processing using separate I/O and Processing threads
- Threading model for task based processing using Connection Partitioned I/O and a Processing Pool
- Multiprocessing server model
- C++ Server Class Hierarchy
- Java Server Class Hierarchy
- Python Server Class Hierarchy
- Apache Thrift Servers depend on abstract protocol factories and abstract protocols, users supply concrete protocol factories which manufacture concrete protocols
- Building I/O stacks with Factories
- Servers use a single handler for service implementation by default
- Processor and Handler Factories allow servers to create per connection handlers
- Apache Thrift Client using a shared I/O stack for input and output
- Servers create separate input and output protocol/transport stacks for use with processors
- Using separate input and output I/O stacks to provide custom output behavior
- TThreadManager task queue and worker thread pool
- TThreadPoolServer connection processing
- Programs, Servers and Services
- Multiplexed Processors and Multiplexed Protocols allow a single server to host multiple services
Summary
While the most important server features and concepts have been covered in this chapter there are still many areas to explore. For example, the top shelf C++ server, TNonblockingServer, is demonstrated in the Part 3 chapter on C++. Part 3 also includes more Java and Python server examples.
Apache Thrift servers provide a rich set of features enabling a wide range of server applications. Here are the key points from our Chapter 10 Server exploration:
- Each language has its own strengths and weaknesses impacting server design and implementation
- Server concurrency is a complex subject and many processing models are represented by the Apache Thrift server library
- Apache Thrift servers supply either connection based or task (RPC Call) based processing models
- Factories are used by servers to generate transports, protocols, processors, handlers and threads in response to client requests
- RPC Clients and Servers maintain separate input and output protocols and transport layers above the network end point
- Users can define many possible I/O processing paths using built-in and custom transports and factories
- Handlers are per server by default but can be generated per connection by supplying servers with a processor and handler factory
- Server events allow user code to intercept client connect, disconnect and processing events in a service independent fashion
- Nonblocking servers require clients to use a framing transport layer
- A single server and port can host many services by using a multiplexed processor
- Multiplexed services can only communicate with clients which make use of multiplexed protocols
FAQ
What does an Apache Thrift server do?
It hosts one or more IDL-defined services, listens for client connections, decodes incoming RPC requests via a protocol/transport stack, dispatches them to your handler implementation, encodes responses, and manages concurrency and lifecycle (accepting, processing, and closing connections).How does the simplest single-threaded server process requests?
It blocks on accept() to get a client transport, wraps it in the chosen transport/protocol stack, then loops calling processor.process(inProto, outProto). When the client disconnects, a transport exception breaks the loop and the server returns to accept() for the next client.What’s the difference between connection-based and task-based server models?
- Connection-based: one thread drives all I/O and calls for a single connection; simple and deterministic per connection, but can create many threads at scale.- Task-based: work is split per RPC “task” and processed by worker threads; scales to many connections with fewer threads but requires framing and careful I/O handoff.
Which server type should I choose?
- TSimpleServer: one client at a time; great for private, low-overhead links.- TThreadedServer: a thread per connection; simple and scalable to moderate counts.
- TThreadPoolServer: threads are pooled and reused per connection; avoids create/destroy overhead.
- Nonblocking/task-based (e.g., C++ TNonblockingServer; Java TNonblockingServer/THsHaServer/TThreadedSelectorServer): high scalability; requires framed transport; tune I/O and worker threads.
- Python: for true concurrency use TForkingServer or TProcessPoolServer due to the GIL.
Programmer’s Guide to Apache Thrift ebook for free
