Appendix D. Using larger LLMs

This appendix explains that the from-scratch Qwen3 implementation used for the small 0.6B model can also load larger checkpoints with only minor adjustments. The main steps are to choose the correct configuration dictionary, download the desired model weights, and use the right tokenizer for either the base or reasoning version. A 4B model is used as the main example because it is significantly stronger than 0.6B while still being more manageable than the larger variants.

The larger dense Qwen3 models follow the same overall architecture pattern as the 0.6B version, including token embeddings, grouped-query attention, rotary position embeddings, feed-forward layers, and RMS normalization. What changes are the scale-related dimensions, such as embedding size, layer count, attention heads, and hidden dimensions. The appendix also gives rough memory estimates for loading these models, emphasizing that actual runtime memory is higher than the raw parameter size because of activations, buffers, and the KV cache.

To load a larger base model, the workflow is to download the Hugging Face snapshot, instantiate Qwen3Model with the matching configuration, map the weights into the custom implementation, and prepare the base tokenizer before generating text. For reasoning variants, the architecture stays the same but the repository, tokenizer filename, and tokenizer settings change so that chat formatting and thinking behavior are enabled. The main takeaway is that scaling up within the same model family is straightforward once the loading pipeline is in place, and the inference code remains essentially unchanged.

Figure D.1 Comparison of different Qwen3 variants, from 0.6-billion to 32-billion parameters.

FAQ