14 Checkpoint Merging

Checkpoint merging is presented as a fast, low-VRAM way to create new Stable Diffusion models by combining the weights of existing checkpoints. The chapter explains that model files are largely just collections of weights, so merging usually means averaging those weights, often with a weighted ratio rather than an even split. This simple idea is compared to mixing paint colors: by adjusting the proportions, you can blend the strengths of two models into a new style without the cost of full training.

The chapter first demonstrates the idea in Automatic1111 using Stable Diffusion 1.5 checkpoints, merging a more stylized model with a more realistic one to produce a hybrid look. The interface makes the process straightforward: choose the primary and secondary models, set the multiplier to control how much each contributes, and run the merge. The resulting checkpoint is saved automatically and can be used right away for image generation, letting users rapidly test different ratios until they find a mix that fits their goals.

It then expands to Stable Diffusion XL in ComfyUI, where the workflow is more complex but far more flexible. ComfyUI allows the user to generate preview images from the original models and the merged model in one run, optionally skip saving the large merged checkpoint, and reuse shared components like prompts and latents to save time. The main takeaway is that checkpoint merging gives creators an efficient form of practical fine-tuning, and ComfyUI makes that experimentation even more powerful through customizable workflows and quick iteration.

A surprising number of models on Hugging Face and CivitAI are the result of merging checkpoints.

Visualizing how weights are used in Neural Networks.

The nodes we visualize represent a tiny portion of the neural network as a whole

Model merging just means averaging the values of the weights in the same locations.

Very often we use a weighted average to merge two models.

CoffeeMix creates a more “cartoony” style image while Deliberate is more realistic.

The checkpoint merge tab in A1111 we haven’t used yet so far.

Setting up our primary (CoffeeMix) and secondary (Deliberate) models

Setting our multiplier to 0.2 we give 80% of the weight to the primary model.

A view of the Checkpoint Merger tab, pushing the “Merge” button is all we need to do now.

Our newly created checkpoint is already in the correct place to be loaded for image generation.

Results of an 80% CoffeeMix, 20% Deliberate checkpoint merge.

Visualizing a range of results by combining CoffeeMix with Deliberate at different ratios.

As is common with ComfyUI workflows, there’s a lot going on here.

We are generating 3 images with each run of the workflow.

Results as seen in ComfyUI.

• Checkpoint merging is a fast and easy way to combine the qualities of two different Stable Diffusion models.
• We merge checkpoints by averaging the model weights between them.
• Typically I want to use a weighted average that gives more weight to one model over the other.
• A1111 makes it very easy to iterate quickly and remains a great tool for fast exploration of Stable Diffusion 1.5 checkpoint merging.
• For more serious work with checkpoint merging it can be helpful to create an entirely custom workflow using ComfyUI.
• Ctrl+M allows us to easily ‘mute’ any node in a ComfyUI work flow which can be very helpful for times we might not want to automatically save large files.
• Despite the simplicity and ease of the technique, checkpoint merging is quite capable as well. Checkpoint merging is responsible for a large number of existing models shared by the community.

FAQ