In the summer of 1997, I was terrified. Instead of working as an intern in my major (microelectronic engineering), the best job I could find was at a research laboratory devoted to high-speed signal processing. My job was to program the two-dimensional fast Fourier transform (FFT) using C and the Message Passing Interface (MPI), and get it running as quickly as possible. The good news was that the lab had sixteen brand new SPARCstations. The bad news was that I knew absolutely nothing about MPI or the FFT.

Thanks to books purchased from a strange new site called Amazon.com, I managed to understand the basics of MPI: the application deploys one set of instructions to multiple computers, and each processor accesses data according to its ID. As each processor finishes its task, it sends its output to the processor whose ID equals 0.

It took me time to grasp the finer details of MPI (blocking versus nonblocking data transfer, synchronous versus asynchronous communication), but as I worked more with the language, I fell in love with distributed computing. I loved the fact that I could get sixteen monstrous computers to process data in lockstep, working together like athletes on a playing field. I felt like a choreographer arranging a dance or a composer writing a symphony for an orchestra. By the end of the internship, I coded multiple versions of the 2-D FFT in MPI, but the lab’s researchers decided that network latency made the computation impractical.

Since that summer, I’ve always gravitated toward high-performance computing, and I’ve had the pleasure of working with digital signal processors, field-programmable gate arrays, and the Cell processor, which serves as the brain of Sony’s PlayStation 3. But nothing beats programming graphics processing units (GPUs) with OpenCL. As today’s supercomputers have shown, no CPU provides the same number-crunching power per watt as a GPU. And no language can target as wide a range of devices as OpenCL.

When AMD released its OpenCL development tools in 2009, I fell in love again. Not only does OpenCL provide new vector types and a wealth of math functions, but it also resembles MPI in many respects. Both toolsets are freely available and their routines can be called in C or C++. In both cases, applications deliver instructions to multiple devices whose processing units rely on IDs to determine which data they should access. MPI and OpenCL also make it possible to send data using similar types of blocking/non-blocking transfers and synchronous/asynchronous communication.

OpenCL is still new in the world of high-performance computing, and many programmers don’t know it exists. To help spread the word about this incredible language, I decided to write OpenCL in Action. I’ve enjoyed working on this book a great deal, and I hope it helps newcomers take advantage of the power of OpenCL and distributed computing in general.

As I write this in the summer of 2011, I feel as though I’ve come full circle. Last night, I put the finishing touches on the FFT application presented in chapter 14. It brought back many pleasant memories of my work with MPI, but I’m amazed by how much the technology has changed. In 1997, the sixteen SPARCstations in my lab took nearly a minute to perform a 32k FFT. In 2011, my $300 graphics card can perform an FFT on millions of data points in seconds.

The technology changes, but the enjoyment remains the same. The learning curve can be steep in the world of distributed computing, but the rewards more than make up for the effort expended.