MLX LM

MLX LM is a Python package for generating text and fine-tuning large language models on Apple silicon with MLX.

MLX LM enables text generation and fine-tuning of large language models on devices equipped with Apple silicon. The library integrates with the Hugging Face Hub, allowing users to download, quantize, and upload models via a command-line interface or Python API. It supports low-rank and full model fine-tuning, as well as distributed inference. Tools included in the package assist with scaling, such as rotating key-value caching and prompt caching for efficient long-context generation. Developers use this library to manage model parameters, sampling, and streaming responses directly within Python applications.

MLX LM at a glance

Best for: Software engineer
Runs on: CLI, macOS
Pricing: Free
Released: March 2025
Category: AI coding tool
Website: github.com/ml-explore/mlx-lm

Key features

Hugging Face Hub integration
Low-rank and full model fine-tuning
Model quantization and conversion
Streaming text generation
Rotating key-value cache
Prompt caching
Distributed inference

Use it for

Generating text from large language models
Fine-tuning models on Apple silicon
Quantizing models for deployment
Interacting with LLMs via a chat REPL

Alternatives to MLX LM

MTPLX — It runs local models twice as fast via native MTP speculative decoding.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
Rapid-MLX — Run 250 MLX-native models locally on Apple Silicon with high inference speeds.
MLC LLM — Enables native optimization and deployment of AI models across platforms.
oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.
Mesh LLM — Run large local models across multiple machines through one OpenAI-compatible endpoint.
LLM CLI — It enables command-line interaction with remote APIs and local machine models.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
LightLLM — It delivers high-speed inference and serving performance for large language models.
LangChain4j — Integrates LLMs and vector stores directly into Java applications.

All MLX LM alternatives