Appendix C. Practical Issues

Practical post-training at scale is dominated by experimentation overhead, so the main lesson is that the real cost is usually not the final successful run but the process of discovering the recipe itself. Developing a strong post-training pipeline can require far more compute than the eventual production training, often by an order of magnitude or more, and it also demands repeated evaluation, debugging, and engineering work. As examples, teams may run thousands of experiments before settling on a final model, and even after that, finalization requires extra time for careful validation and confirmation.

A major challenge is that post-training is operationally noisy: different stages are run multiple times, hyperparameters are swept, and checkpoint evaluation itself can consume a large share of resources, especially for reasoning models that require longer sampled generations. Training runs may be short in pure compute but still stretch over days or weeks because of instability, lost work, or the need to continue improving a model after release. In this setting, time-to-result can matter more than raw compute, since refining a model with reinforcement learning or similar methods may require substantial additional wall-clock investment.

Evaluation and training both exhibit meaningful variance, so good practice requires explicitly managing randomness rather than assuming single-run results are reliable. Some benchmarks are highly unstable while others are comparatively consistent, and noisy evaluations can be stabilized by repeating them, though that increases cost. Training uncertainty is even more consequential because it cannot be averaged away after the fact, so teams typically sweep key optimization settings, run multiple seeds, and use techniques like model merging to capture beneficial outliers and maximize the final model’s quality.