This chapter introduces Apache Thrift’s Interface Definition Language (IDL) as a concise way to describe cross-language service contracts and data shapes. It explains how IDL separates interface mechanics from implementation details, letting the Thrift compiler generate serialization and RPC scaffolding in many languages. Interfaces are modeled as user-defined types and services; the IDL captures what is exchanged and how, while non-mechanical semantics are clarified through documentation. The chapter emphasizes clarity, explicitness, and evolvability so distributed systems can interoperate reliably over time.
Core language features and conventions are surveyed: file naming and element naming rules, keywords and reserved words, and the use of namespaces to avoid global collisions. The type system spans base types, containers (list, set, map), literals, typedefs, and enums, along with compound constructs—structs, unions, exceptions, and argument lists. Fields carry numeric IDs and requiredness (required, default, optional), may have defaults, and support forward/backward compatibility by ignoring unknown fields. The chapter covers constants, comments and doc strings, service definitions (functions, oneway calls, throws clauses), and best practices such as assigning explicit field IDs, expressing semantics via types, keeping interfaces free of implementation details, and using documentation for constraints the syntax cannot enforce.
On the tooling side, the Thrift compiler’s scanning, parsing, and code generation phases are described, with guidance on interpreting errors, selecting output languages with -gen, enabling recursion with -r, and managing include paths with -I. It notes language-specific nuances (for example, container ordering, enum representations, optional handling), customization hooks (cpp_type for C++ containers, sorted_containers for Java), documentation generation, and annotations like cpp.type and final for targeted codegen behavior. The chapter closes with patterns for modular IDL organization via includes, filename scoping of imported symbols, and practical advice—use stable namespaces, alphanumeric .thrift filenames, avoid reserved words, treat map/set ordering as undefined, and design for schema evolution—so teams can produce durable, maintainable cross-language APIs.
Apache Thrift IDL.
Apache Thrift IDL Compiler compilations phases.
Graphviz IDL output.
IDL Compiler HTML output.
IDL file includes.
IDL include hierarchy.
Summary
Apache Thrift IDL is an expressive yet compact interface definition language. It provides modern features while supporting a wide range of implementation languages.
- IDLs support the process of developing explicit mechanical contracts between clients and servers.
- Apache Thrift supports a selection of commenting styles including doc strings that can be used to generate documentation with the Apache Thrift IDL Compiler and other tools (that is, Doxygen).
- Apache Thrift IDL supports a small but flexible set of base types:
- o binary
- o bool
- o byte
- o double
- o i16
- o i32
- o i64
- o string
- o void
- Apache Thrift IDL supports three container types:
- o list
- o set
- o map
- Apache Thrift IDL supports interface constants.
- Apache Thrift IDL supports several user defined types:
- o typedef
- o enum
- o struct
- o union
- o exception
- Apache Thrift IDL doesn’t support type inheritance.
- Apache Thrift IDL doesn’t support self-referential types or forward definitions (with experimental exceptions).
- The service keyword allows RPC service interfaces to be defined.
- Apache Thrift supports interface inheritance but not overloading or overriding.
- IDL files can include other IDL files allowing large interfaces to be organized across files.
- The namespace keyword supports namespace and package generation in various target languages.
FAQ
What is Apache Thrift IDL and what problems does it solve?
Apache Thrift IDL is a language-neutral way to describe interfaces—types and remote services—so tools can generate cross-language serialization and RPC code for you. It makes the “mechanics” of interop (wires, formats, stubs) explicit and consistent, so teams can focus on domain semantics. Even with REST/JSON, you still need clear contracts; IDL provides concise mechanical contracts and space for doc strings to capture additional semantics.How should I name IDL files and elements to avoid issues?
- Use the .thrift extension (some tools require it).- Stick to alphanumeric file names; odd characters can produce illegal identifiers in generated code.
- Element names are case-sensitive, must match the pattern [a-zA-Z_][\.a-zA-Z_0-9]*, and must be unique at the top level regardless of kind (no service and struct with the same name).
- Don’t use keywords or reserved words from Thrift or target languages.
How does the Thrift compiler work, and what are the key command-line flags?
- Phases: scanner (tokens), parser (grammar to program elements), generators (emit code for selected languages).- Typical flags: -gen lang (one or more output languages), -r/--recurse (also generate for included IDL), -I path (include search path), -o/--out (output directory), -v (verbose), -strict (stricter checks).
- Error sources: scanner errors (bad tokens, bad escapes), parser errors (grammar/semantics like duplicate field IDs), and rare generator-time issues (e.g., OOM).
How do I organize IDL across files and languages (namespaces and includes)?
- Use namespace directives to place generated code into language-specific packages/modules; namespace * Foo applies to all supported targets and is a best practice.- Namespace lines must appear before definitions; later namespace lines override earlier ones for their scope.
- Include other IDL files with include "file.thrift". To reference their types, prefix with the file name (e.g., track.MusicTrack). Includes are not transitive—you must include each file you directly use.
- Use -r to generate code for dependencies and -I to add include search paths.
What built-in types and containers does Thrift provide?
- Base types: bool, i8/i16/i32/i64, double, string, binary, and void (only as a return type).- Containers: list
- Notes: strings are UTF-8 on the wire; binary is raw bytes. Prefer set
How do constants, typedefs, enums, and literals work (and what are best practices)?
- Constants: Declare with const; they become language-appropriate constants/classes in generated code. Put only contract-relevant constants in IDL.- Typedefs: Create aliases (e.g., typedef map
- Enums: Define discrete sets; assign explicit, positive values to avoid compatibility surprises. Initialize using EnumType.Value for readability.
- Literals: Support for ints (decimal/hex), doubles (incl. scientific notation), strings/binary with escapes, and container literals. Use true/false for booleans (avoid numeric truthiness). Ensure literal sizes fit target types (IDL won’t warn about overflow).
What are structs, unions, exceptions, and argument lists—and how do they differ?
- All collect fields and get read()/write() methods in generated code.- Structs: general-purpose UDTs (most common).
- Unions: exactly one field valid at a time; no required fields; serialized as a struct; some languages emulate union semantics—set only one field yourself if not enforced.
- Exceptions: declared like structs but integrate with the language’s exception hierarchy; service methods can expose them via throws.
- Function argument lists: syntactically like struct fields (IDs, defaults), but no required/optional modifiers (they’re default-requiredness for writing behavior).
How do field IDs, requiredness, and default values affect compatibility?
- Field IDs: Use explicit, positive IDs; they’re how fields are matched on the wire. Relying on implicit (negative) IDs ties layout to field order and breaks compatibility when reordering.- Requiredness: required (must be present), default (always written; read if present), optional (written only if set). Unknown fields are ignored by readers.
- Defaults: For required/default fields, both writer and reader initialize defaults; transmitted values overwrite. Optional fields with defaults may or may not be transmitted depending on language—avoid optional defaults unless you’ve validated behavior across your targets.
How do services and functions work (returns, oneway, exceptions)?
- Services define RPC interfaces; only services support inheritance via extends (single inheritance; no overrides/overloads of inherited methods).- Functions can return any non-exception type or void and take ID-annotated parameters; arguments can have default values.
- oneway void methods send no response—lower latency/overhead but no completion or error feedback.
- throws lists declare which user-defined exceptions may be sent back; framework exceptions are also possible.
What annotations exist and when should I use them?
- Annotations are key/value hints some generators honor (others ignore).- Notable ones: (cpp.type="...") to override emitted C++ base type for a field; (final="true") to prevent subclassing (AS3/C#/Delphi/Java) or suppress virtual destructor (C++).
- Use annotations sparingly for language-specific needs; they keep the core IDL generic while enabling targeted customization.
Programmer’s Guide to Apache Thrift ebook for free
