7 Deterministic forecasting with TimesFM

This chapter introduces deterministic (point) forecasting and TimesFM, a foundation model from Google Research that outputs single-value forecasts instead of full predictive distributions. The authors contrast this approach with probabilistic models used earlier in the book, noting that while point forecasts simplify downstream use and suit planning scenarios, they do not quantify uncertainty and therefore are not appropriate for tasks like anomaly detection. Although TimesFM can emit experimental quantiles, these are not calibrated and should not be relied upon for interval-based decisions. The chapter focuses on TimesFM’s architecture and pretraining protocol, shows how to apply it in zero-shot mode, and explores how to incorporate exogenous features.

Architecturally, TimesFM is a decoder-only Transformer that processes non-overlapping input patches and is trained to produce longer output patches than inputs, reducing the number of autoregressive steps required for long horizons. Residual MLP blocks embed patched inputs and map Transformer outputs back to forecasts, with positional encodings added as in standard Transformers. A random input-patch masking strategy during pretraining enables handling varied context lengths. The model is trained with mean squared error, and was pretrained on a massive corpus exceeding 380 billion points, dominated by web traffic data (e.g., Wiki Pageviews, Google Trends), plus synthetic series (including ARMA, seasonal sinusoids, trends, step functions). Practical constraints include maximum context lengths (initially up to 512 steps, extended to 2048 in a later 500M-parameter release), autoregressive generation beyond the 128-step output patch, substantial memory needs (around 16GB RAM), and open-source availability primarily for the largest checkpoints.

In practice, TimesFM is straightforward for zero-shot forecasting via a DataFrame API that expects standardized column names and a frequency setting; cross-validation must be implemented manually. On a weekly retail example, zero-shot performance was competitive—outperforming Moirai and approaching fine-tuned baselines—though it struggled with abrupt level shifts. Exogenous features are supported outside the core model via a simple ridge regression wrapper that combines covariate effects with TimesFM’s forecasts (either modeling the target first and covariate-driven residuals second, or vice versa). This requires future covariate values; in the example, a holiday indicator helped capture peaks but degraded overall error metrics. Fine-tuning presently demands JAX expertise and custom training loops, and given its deterministic nature (and uncalibrated quantiles), TimesFM is not suitable for anomaly detection.

Architecture of TimesFM. The input series is patched and set to a residual block that maps the input patches to a vector. The residual block is simply a multilayer perceptron with one hidden layer and a skip connection. The output vector is added to positional encoding and sent to the decoder-only Transformer. The output of the Transformer is sent to another residual block which returns a sequence of predicted values.

In TimesFM, the output patch is longer than the input patch. That way, the model performs less autoregressive steps when forecasting longer horizons, which results in better performance.

Input patch masking strategy during the training phase of TimesFM. The input patch is randomly masked, leaving at least one time step. That way, the model is trained to output an output patch with a varying input length.

Focusing on the decoder-only portion of TimesFM. The decoder is made of many decoder layers, each using multi-headed attention.

Zero-shot predictions of the weekly sales for each store. The shaded area shows the 90% prediction interval, but it comes from experimental quantiles that were not calibrated, so it may not be valid.

Predictions obtained from cross-validation with TimesFM. Note that the shaded area represents a 90% prediction interval obtained from experimental quantiles that were not calibrated, so it may not be valid.

There are two options for handling exogenous features in TimesFM. In option 1, we forecast the target itself first with TimesFM, then model the residuals with the exogenous features, and add both predictions for the final forecast. In option 2, we first model the exogenous features using linear regression, then TimesFM forecasts the residuals, and both outputs are added for the final predictions.

Comparing the forecasts obtained with and without covariates. We notice that by using covariates, the model predicts future peaks in sales.

MAE and sMAPE for forecasting with covariates and without covariates. Lower is better. Here, using covariates results in less accurate forecast from TimesFM.

• TimesFM is a deterministic foundation model developed by Google Research. It uses a decoder-only Transformer for point forecasting.
• The model is open-source and free to use. However, only the largest version is available.
• It supports exogenous features by using an external linear regression algorithm.
• Since it is deterministic, it cannot perform anomaly detection.
• Fine-tuning the model requires knowledge of JAX, making it inaccessible to practitioners unfamiliar with the technology.

FAQ