MergeKit

MergeKit is a toolkit for merging pre-trained language models using an out-of-core approach.

MergeKit is a toolkit designed for combining pre-trained language models directly in the weight space without additional training. It supports various architectures like Llama, Mistral, and GPT-NeoX, enabling users to combine specialized models, transfer capabilities, and create mixture of experts or frankenmerzed models. The software operates using YAML configuration files via a command-line script or an in-memory Python API. It features lazy loading of tensors for low memory usage and supports both CPU and GPU execution. It is intended for developers and researchers looking to optimize model performance and share results on the Hugging Face Hub.

MergeKit at a glance

Best for: Software engineer, Researcher
Runs on: CLI, self-hosted, open source
Pricing: Free
Released: August 2023
Category: AI coding tool
Website: github.com/arcee-ai/mergekit

Key features

Supports Llama, Mistral, GPT-NeoX, and StableLM
Multiple merge methods
GPU or CPU execution
Lazy loading of tensors for low memory use
Piecewise assembly of language models from layers
Mixture of Experts merging
LoRA extraction
Evolutionary merge methods

Use it for

Combine multiple specialized models into a single versatile model
Transfer capabilities between models without training data
Find optimal trade-offs between different model behaviors
Create new capabilities through creative model combinations

Alternatives to MergeKit

LMDeploy — Accelerates inference with high-performance engines and quantization tools.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
Genkit — Provides unified APIs and local developer tools for AI application workflows.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
GPUStack — Unifies AI model serving and GPU compute management across heterogeneous hardware.
MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
Enki LLM — Runs Llama-3.2 1B in pure Rust at 40 tok/s with live GPU to CPU migration.
MTPLX — It runs local models twice as fast via native MTP speculative decoding.
LlamaFactory — Provides a code-free CLI and Web UI for training 100+ different LLMs.
llmware — Build private, local RAG pipelines with 300+ prepackaged models and integrated document parsing.

All MergeKit alternatives