Why merge models instead of diving straight into training?
If you work with model series like Llama 3 or Qwen, you’ve likely encountered a situation where Model A writes code perfectly, but Model B speaks Vietnamese more naturally. You long for a “hybrid” that possesses both strengths.
The traditional path is Fine-tuning. However, in reality, this is extremely expensive. Renting an H100 GPU cluster currently costs about $4-$5 per hour. Not to mention, you might face “catastrophic forgetting” – where the model learns new skills but forgets its original knowledge.
Model Merging appears as a smart shortcut. Instead of re-teaching from scratch, we blend the “weights” of existing models using algorithms. This process costs almost zero GPU expense. You can perform it right on standard RAM or a CPU.
On the Scale: Merging vs Fine-tuning
Based on my experience implementing real-world chatbot projects, I’ve summarized the differences to help you choose:
- Fine-tuning: Like sending an employee to an intensive course. Costs tuition (GPU), takes time, and carries a high risk of “rote learning” (overfitting).
- Model Merging: Like combining the experience of two experts. Performed lightning-fast in 10-30 minutes, preserves original intelligence, and costs nearly zero.
I once applied this to a RAG (Retrieval-Augmented Generation) system. Instead of expensive fine-tuning, I merged a conversation-specialized version with an information-extraction version. The result was accurate responses without repetitive words or loss of context.
3 Most Popular Model “Cooking” Algorithms
MergeKit supports many methods, but these are the 3 names you must know:
1. SLERP (Spherical Linear Interpolation)
This is the go-to choice for blending two models with the same structure. Instead of a crude average, SLERP calculates based on the angle between vectors. This helps retain the unique characteristics of each model without distorting the weights.
2. TIES (Trimming, Electing, and Merging)
When you want to mix 3 or more models, use TIES. This algorithm is smart because it automatically removes small changes (noise) and keeps only the essentials, effectively resolving conflicts between component models.
3. DARE (Drop and Rescale)
DARE takes a bolder approach by removing redundant weights. Merged models are often very compact and offer performance close to multi-task fine-tuning.
Deploying MergeKit on Linux
You need a Linux server, ideally Ubuntu 22.04. Regarding hardware, GPU isn’t very important, but RAM must be large. In my experience, RAM should be larger than the total size of the input models. To merge two 7B models (about 15GB each), you should prepare at least 32GB to 64GB of RAM.
Step 1: Set up the environment
Never install directly into the system. Always use a virtual environment to avoid library conflicts.
# Update system
sudo apt update && sudo apt upgrade -y
# Install python venv
sudo apt install python3-venv -y
# Create and activate virtual environment
python3 -m venv mergekit_env
source mergekit_env/bin/activate
# Install MergeKit from source
git clone https://github.com/arcee-ai/mergekit.git
cd mergekit
pip install -e .Step 2: Write the Recipe (YAML file)
The YAML file is where you define this “marriage.” Suppose we merge two models from the Llama-3-8B family to optimize Vietnamese capability and reasoning.
slices:
- sources:
- model: meta-llama/Meta-Llama-3-8B-Instruct
layer_range: [0, 32]
- model: NousResearch/Hermes-2-Theta-Llama-3-8B
layer_range: [0, 32]
merge_method: slerp
base_model: meta-llama/Meta-Llama-3-8B-Instruct
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: bfloat16Small tip: Models must have the same layer structure and hidden size. You cannot mix a 7B model with a 70B model this way.
Step 3: Proceed with Merging
Everything is ready. Run the following command for MergeKit to start its magic:
mergekit-yaml config.yaml ./my-merged-model --copy-tokenizer --allow-crimesQuick explanation of flags:
./my-merged-model: Where to save the final product.--copy-tokenizer: Retain the tokenizer from the base model.--allow-crimes: Allows processing if there are minor structural discrepancies (use with caution).
Step 4: Verify the Results
Once finished, the output directory will contain .safetensors files. You can use the transformers library to quickly test the response of the new model.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_path = "./my-merged-model"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto")
# Test a challenging question
inputs = tokenizer("Explain the theory of relativity in simple Vietnamese", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))Lessons Learned from Merging Models
Despite its power, Model Merging is not a “magic key.” After many trials and errors, I’ve drawn a few points to note:
Pros:
- Create new models extremely fast on high-RAM VPS.
- Maximize power from the Hugging Face community.
- Flexible customization via YAML configuration.
Cons:
- Results can sometimes be hit-or-miss. Models can act “confused” if components conflict too strongly.
- Time-consuming to fine-tune parameters (ratio t) for peak performance.
- Chat Template errors frequently occur if tokenizers are inconsistent.
A Message for Developers
MergeKit is a game-changer for those who want to do AI properly but have a tight budget. Instead of a GPU arms race, focus on selecting and combining models intelligently.
Hãy bắt đầu với 2 model nhỏ 7B hoặc 8B bằng phương pháp SLERP. Khi đã quen tay, bạn có thể tiến tới các phương pháp phức tạp như TIES để tạo ra những “quái vật” thực thụ trên bảng xếp hạng Open LLM Leaderboard.
