Overview

Chapter 9. Distributed tracing with Spring Cloud Sleuth and Zipkin

This chapter explains why distributed systems need better observability than a single-service application. In a microservices environment, requests cross multiple services, machines, and data stores, so debugging and performance analysis become difficult without a way to connect related activity. The chapter introduces three complementary techniques for solving this problem: tracing requests with correlation IDs, centralizing logs so they can be searched in one place, and visualizing request flow with distributed tracing tools.

Spring Cloud Sleuth is presented as the easiest way to add tracing to Spring microservices. By adding a single dependency, services automatically create and propagate trace information, include it in logs through the logging context, and pass it along to downstream HTTP calls and messaging channels. The chapter shows how this replaces much of the manual infrastructure that would otherwise be needed, including custom filters, interceptors, and thread-context handling. It also explains the structure of the trace data, including application name, trace ID, span ID, and whether data is sent onward to Zipkin.

The chapter then extends tracing into centralized log aggregation and visual distributed tracing. Using Papertrail as an example, it shows how container output can be routed through Logspout into a searchable logging service, making it easy to find all log entries for a transaction by trace ID. Finally, it introduces Zipkin for end-to-end trace visualization, showing how to configure services and the Zipkin server, adjust sampling, and inspect spans across HTTP calls and messaging flows. The chapter closes by demonstrating custom spans, which let developers measure calls to external systems such as Redis and PostgreSQL so that hidden latency hotspots become visible in the trace.

Figure 9.1. Spring Cloud Sleuth adds four pieces of tracing information to each log entry written by your service. This data helps tie together service calls for a user’s request.

Figure 9.2. With multiple services involved in a transaction, you can see that they share the same trace ID.

Figure 9.3. The combination of aggregated logs and a unique transaction ID across service log entries makes debugging distributed transactions more manageable.

Figure 9.4. Using native Docker capabilities, logspot, and Papertrail allows you to quickly implement a unified logging architecture.

Figure 9.5. To begin, create an account on Papertrail.

Figure 9.6. Next choose how you’re going to send log data to Papertrail.

Figure 9.7. Papertrail uses Syslog as one of the mechanisms for sending data to it.

Figure 9.8. With the Logspout Docker container defined, data written to each container’s standard out will be sent to Papertrail.

Figure 9.9. The trace ID allows you to filter all log entries related to that single transaction.

Figure 9.10. With the Spring Cloud Sleuth trace ID returned, you can easily query Papertrail for the logs.

Figure 9.11. The Zipkin query screen lets you select the service you want to trace on, along with some basic query filters.

Figure 9.12. Zipkin allows you to drill down and see the amount of time each span in a transaction takes.

Figure 9.13. Clicking on an individual span gives further details on call timing and the details of the HTTP call.

Figure 9.14. Viewing the details of a trace of how the licensing service call flows from Zuul to the licensing service and then through to the organization service

Figure 9.15. With the trace ID returned in the HTTP Response tmx-correlation-id field you can easily find the transaction you’re looking for.

Figure 9.16. Spring Cloud Sleuth will automatically trace the publication and receipt of messages on Spring message channels.

Figure 9.17. You’re looking for the licensing service invocation where a Kafka message is received.

Figure 9.18. Using Zipkin you can see the Kafka message being published by the organization service.

Figure 9.19. With the custom spans defined, they’ll now show up in the transaction trace.

Summary

  • Spring Cloud Sleuth allows you to seamlessly add tracing information (correlation ID) to your microservice calls.
  • Correlation IDs can be used to link log entries across multiple services. They allow you to see the behavior of a transaction across all the services involved in a single transaction.
  • While correlation IDs are powerful, you need to partner this concept with a log aggregation platform that will allow you to ingest logs from multiple sources and then search and query their contents.
  • While multiple on-premise log aggregation platforms exist, cloud-based services allow you to manage your logs without having to have extensive infrastructure in place. They also allow you to easily scale as your application logging volume grows.
  • You can integrate Docker containers with a log aggregation platform to capture all the logging data being written to the containers stdout/stderr. In this chapter, you integrated your Docker containers with Logspout and an online cloud logging provider, Papertrail, to capture and query your logs.
  • While a unified logging platform is important, the ability to visually trace a transaction through its microservices is also a valuable tool.
  • Zipkin allows you to see the dependencies that exist between services when a call to a service is made.
  • Spring Cloud Sleuth integrates with Zipkin. Zipkin allows you to graphically see the flow of your transactions and understand the performance characteristics of each microservice involved in a user’s transaction.
  • Spring Cloud Sleuth will automatically capture trace data for an HTTP call and inbound/outbound message channel used within a Spring Cloud Sleuth enabled service.
  • Spring Cloud Sleuth maps each of the service call to the concept of a span. Zipkin allows you to see the performance of a span.
  • Spring Cloud Sleuth and Zipkin also allow you to define your own custom spans so that you can understand the performance of non-Spring-based resources (a database server such as Postgres or Redis).

FAQ

What problem does distributed tracing solve in a microservices architecture?It helps you follow a single user transaction across multiple services, machines, and data stores so you can debug failures and performance issues more easily.
What is a correlation ID and how is it used in this chapter?A correlation ID is a unique identifier assigned to a transaction and propagated across service calls so logs from different services can be linked together.
How does Spring Cloud Sleuth reduce the manual work of adding correlation IDs?Spring Cloud Sleuth automatically creates or detects tracing information, propagates it to outbound calls, and adds it to Spring logging context so you do not need to build all that infrastructure yourself.
What Maven dependency do you add to enable Spring Cloud Sleuth in a service?You add the spring-cloud-starter-sleuth dependency to the service’s pom.xml.
What information does Sleuth add to each log entry?It adds the application name, the trace ID, the span ID, and a flag indicating whether trace data was sent to Zipkin.
Why is log aggregation important when tracing distributed transactions?Because logs are spread across many service instances, a centralized log platform lets you search by trace ID and quickly reconstruct what happened in a transaction.
Why does the chapter use Papertrail with Logspout for log aggregation?Papertrail provides an easy cloud-based log destination, and Logspout can capture Docker container output and forward it to Papertrail’s syslog endpoint.
How can you make the trace ID visible in an HTTP response?In the chapter, a Zuul post filter uses the Sleuth Tracer to read the current trace ID and add it to a response header named tmx-correlation-id.
What is Zipkin used for in this chapter?Zipkin is used to visually trace requests across services, break transactions into spans, and show how much time each service or call takes.
How do you configure services to send trace data to Zipkin?You add the spring-cloud-sleuth-zipkin dependency and set the spring.zipkin.baseUrl property to point to the Zipkin server.