This chapter introduces timeseries forecasting as the task of predicting future values from regularly sampled observations, emphasizing the dynamics of periodicity, trends, and spikes. It outlines common tasks beyond forecasting—such as anomaly detection, classification, and event detection—then focuses on a concrete problem: predicting temperature 24 hours ahead from multivariate weather measurements. The narrative stresses the importance of inspecting periodic patterns, using temporally ordered train/validation/test splits, and establishing a simple, common-sense baseline to quantify real progress before deploying more complex models.
Using a multiyear weather dataset with 14 variables, the workflow normalizes features and constructs sliding input windows (about five days of hourly-sampled context) to predict the next day’s temperature. A persistence baseline (predict tomorrow equals now) delivers a surprisingly strong mean absolute error around 2.5°C, setting a high bar. Simple dense networks that flatten time underperform because they discard temporal structure, and 1D convnets struggle because weather signals are not strictly translation-invariant and pooling harms order information; both observations underscore the need for models that explicitly respect sequence order and recency.
Recurrent neural networks address this by maintaining state across timesteps; LSTMs, in particular, mitigate vanishing gradients via gated memory and outperform the baseline. The chapter explains how RNNs work conceptually and in Keras, then demonstrates practical upgrades: recurrent dropout to combat overfitting (using time-constant masks), stacking layers to increase capacity (e.g., GRU/LSTM stacks), and when bidirectional RNNs help (notably for text, less so for this task where recent data matters most). It closes with guidance on iteration and tuning—adjusting sequence length, sampling, units, dropout, and optimizers—and notes that, with single-location inputs, gains beyond roughly 10% over the baseline are hard to achieve, reflecting the inherent limits of local weather predictability and the broader lesson that not all timeseries (such as financial prices) are forecastable from past observations alone.
Temperature over the full temporal range of the dataset (ºC)
Temperature over the first 10 days of the dataset (ºC)
Training and validation MAE on the Jena temperature-forecasting task with a simple, densely connected network
Training and validation MAE on the Jena temperature-forecasting task with a 1D convnet
Training and validation MAE on the Jena temperature-forecasting task with a LSTM-based model (note that we omit epoch 1 on this graph, because the high training MAE (7.75) at epoch 1 would distort the scale)
A recurrent network: a network with a loop
A simple RNN, unrolled over time
The starting point of an LSTM layer: a SimpleRNN
Going from a SimpleRNN to an LSTM: adding a carry track
Anatomy of an LSTM
Training and validation loss on the Jena temperature-forecasting task with a dropout-regularized LSTM
Training and validation loss on the Jena temperature-forecasting task with a stacked GRU network
How a bidirectional RNN layer works
FAQ
What are timeseries and which machine-learning tasks can you do with them?Timeseries are measurements taken at regular intervals (for example, hourly temperature or daily sales). Beyond forecasting, common tasks include: anomaly detection (spot unusual behavior in a stream), classification (assign labels to a sequence, like bot vs human), and event detection (flag expected events such as a wake word in audio).How is the chapter’s temperature forecasting problem defined?Given the past five days of hourly observations (from 14 weather variables recorded every 10 minutes and downsampled to 1/hour), predict the temperature 24 hours into the future. The data comes from the Jena climate dataset (2009–2016).How should I split timeseries into training, validation, and test sets?Keep the temporal order: use older data for training and more recent data for validation and testing (for example, first 50% train, next 25% validation, last 25% test). Don’t shuffle across time when creating splits.How do I prepare the data before modeling?Normalize each feature using statistics computed on the training portion only (subtract train mean, divide by train std). Then create sliding windows: for example, sequence_length=120 (5 days of hourly points), sampling_rate=6 (from 10-minute data to 1/hour), and set delay=sampling_rate*(sequence_length+24−1) so the target is temperature 24 hours after the window ends.What does timeseries_dataset_from_array() do, and how do I use it?It turns an array of timesteps into overlapping windows (sequences) and aligns them with targets. Example: for next-step prediction on data=[0,1,2,3,4,...] with sequence_length=3, inputs are [0,1,2], [1,2,3], ... and targets are [3,4,...]. For 24-hour-ahead forecasting, pass raw_data[:-delay] as inputs and temperature[delay:] as targets, along with sampling_rate and sequence_length.What’s a simple non-ML baseline and how well does it work?The persistence baseline: predict that temperature in 24 hours equals the current temperature. On this task it yields roughly 2.5°C mean absolute error (MAE). Any ML model should beat this to be considered useful.Why did basic dense and 1D convolutional models underperform?Dense models require flattening, which discards temporal structure. 1D convnets assume translation invariance and pooling erases order, but for weather, mornings differ from evenings and recent timesteps matter more than older ones. These mismatches limit performance.What are RNNs and LSTMs, and why do they help here?RNNs process sequences step by step while carrying a state. LSTMs add gating to preserve and reuse information over longer spans, mitigating vanishing gradients. On this task, a simple LSTM surpasses the baseline (with MAE modestly below the persistence error).How can I reduce overfitting in recurrent models?Use recurrent dropout (recurrent_dropout) with time-constant masks, optionally add a Dropout layer after the RNN output, and train longer. Note: enabling recurrent dropout disables cuDNN-accelerated kernels on GPU; consider unroll=True for small, fixed-length sequences to speed up CPU/GPU execution.When should I stack or use bidirectional recurrent layers?Stacking (multiple LSTM/GRU layers) increases capacity and can yield small gains until overfitting dominates. Bidirectional RNNs work well when order matters but the direction doesn’t (e.g., many NLP tasks). For this weather task, bidirectional models tend to underperform because recent-to-past (chronological) information is more predictive than reversed time.
pro $24.99 per month
access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
Deep Learning with Python, Third Edition ebook for free
