Many people know about classical algorithms for common tasks like hashing, sorting, searching. But what if the data to handle can’t fit in memory anymore? This book provides the answers!
Data structures and algorithms that are great for traditional software may quickly slow or fail altogether when applied to huge datasets. Algorithms and Data Structures for Massive Datasets introduces a toolbox of new techniques that are perfect for handling modern big data applications. You'll discover methods for reducing and sketching data so it fits in small memory without losing accuracy, and unlock the algorithms and data structures that form the backbone of a big data system. Filled with fun illustrations and examples from real-world businesses, you'll learn how each of these complex techniques can be practically applied to maximize the accuracy and throughput of big data processing and analytics.
about the technology
Modern data-intensive applications are outpacing traditional data structures and algorithms. Huge data sets rapidly grow beyond available memory, becoming slow and inefficient, and bottlenecking development. Fortunately, you don't need to blow your budget on expensive upgrades to your computing power! Algorithms and Data Structures for Massive Datasets lays out ways to sketch data in main memory and organize data on disk to make the best use of your available resources. Taken from the latest research papers, these effective techniques apply to any discipline, from finance to text analysis.
about the book
Algorithms and Data Structures for Massive Datasets
teaches you to take advantage of data processing and analytics techniques specifically designed for large distributed datasets. And you'll be amazed how easy it is to learn such a challenging topic from this friendly guide! Complex concepts are illustrated with interesting, entertaining graphics and fascinating industry stories that show how these techniques have succeeded in the real world. You'll study examples including Google BigTable, BitCoin, and a smart bed sensor app, learning to build data sketches for processing, querying and exploring large datasets. By the time you're done, you'll be able to identify the perfect algorithm to deliver faster and more reliable results for any data intensive system.
- Sketching data structures for practical problems
- Choosing the right database engine for your application
- Evaluating and designing efficient on-disk data structures and algorithms
- Understanding the algorithmic tradeoffs involved in massive-scale systems
- Deriving basic statistics from streaming data
- Correctly sampling streaming data
- Computing percentiles with limited space resources
about the reader
For programmers familiar with fundamental data structures. Examples in R and pseudocode.
about the author
earned her PhD in the Applied Algorithms Lab of the computer science department at Stony Brook University, NY in 2014. She has worked on a number of projects in algorithms for massive data, taught algorithms at various levels and also spent some time at Microsoft. Emin Tahirovic
earned his doctorate in biostatistics from UPenn in 2016, and his master's degree in theoretical computer science from Goethe University in Frankfurt in 2008. He has worked for DBahn AG as an IT consultant and he regularly consults on projects for pharma and tech companies. Ines Dedovic
earned her PhD at the Institute for Imaging and Computer Vision of the Department of Electrical Engineering at RWTH Aachen University, Germany. She has worked as a researcher at the Research Center Jülich and is currently employed as a software developer for camera systems at Jonas & Redmann, an automation company.