7 User-defined types

This chapter deepens the practical use of Apache Thrift by focusing on user-defined types (UDTs) as the core of cross-language data exchange. Building on transports, protocols, and IDL syntax, it shows how to declare rich, portable data models that serialize cleanly across C++, Java, Python, and more, using generated read/write methods. UDTs are equally valuable in RPC and non-RPC workflows—moving safely through message queues, files, or databases—demonstrated with a simple EarthRelPosition example and a more realistic RadioObservation model, all while leveraging Thrift’s pluggable I/O stack.

Effective type design uses Thrift’s core tools—structs, unions, enums, typedefs, base types, and collections—combined with disciplined practices. Field IDs must be positive, unique, and never reused; deleted fields are retired (commented out) to preserve compatibility. Requiredness matters: default is a safe general choice, optional is best for large or situational data (like an image), and required should be used sparingly because it cannot evolve. Unions communicate intentional type variability (for positions captured in different coordinate systems), maps vs. lists capture semantics like uniqueness, typedefs convey meaning (Time), and enums should keep stable numeric values while handling unknowns gracefully. Generated code patterns vary by language—e.g., setter/isset methods in Java, __isset flags in C++, and None checks in Python—but the interoperability contract is consistent.

Under the hood, generated classes include field metadata plus read/write methods that serialize by field ID and type, not name; readers loop, test IDs and types, and skip what they don’t recognize, enabling forward and backward compatibility. The chapter outlines robust evolution strategies: rename fields freely without changing semantics; add fields as default (ideally with defaults) or optional; delete by retiring IDs; avoid changing a field’s type unless you planned for it with a union; and treat changes to requiredness and default values with care. Finally, it shows whole-object compression for storage efficiency using a layered TZlibTransport beneath any protocol, achieving strong size reductions (especially with binary payloads) while preserving cross-language compatibility and tunable compression levels.

Apache Thrift Framework user-defined types.

The Arecibo Radio Telescope is an example RadioObservationSystem that might contribute RadioObservation data for use with the radio_observation.cpp program.

A Graphviz diagram of the radio_observation type interface.

The 25KB quasar.bmp used in the sky_bmp field of the RadioObservation test object created by DiskSer.java.

Programs using different versions of an interface can communicate with proper use of interface evolution in Apache Thrift.

Type serialization Zlib compression stack.

• Structs are the Apache Thrift IDL mechanism for creating cross-language user-defined types (UDTs).
• Structs have one or more fields each with a name, Id, type, requiredness, and optionally, a default value.
• Optional requiredness fields offer the most serialization flexibility allowing user code to decide whether to serialize them or not.
  • o Optional fields are a good choice for any data that may not need to be serialized on all occasions, particularly data fields of large size.
  • o Optional fields must typically be set with language specific set methods to ensure that the UDT serializes them when the write() method is called.
  • o Optional fields must be tested for existence after deserialization and prior to access in case they were not found during the deserialization process.
• Typedefs allow new semantic types to be created from existing types.
• Enums allow new enumeration types to be created.
• Unions are used to create fields that have more than one possible type or representation.
  • o All union fields are optional.
  • o Union values must be set with set methods in most languages.
  • o Only one type should be set at a time within a union.
• UDTs can be serialized by calling their write() method and de-serialized by calling their read() method.
• Apache Thrift Interface Evolution features allow UDTs to change over time without breaking existing applications.
  • o New fields can be added.
  • o Old fields can be removed.
  • o Fields can be represented with a selection of types when unions are used.
• The TZlibTransport can be layered on top of memory and file end point transports to compress serialized objects.
  • o The TZlibTransport isn’t supported by all languages.

FAQ