Carmen

Carmen is a command-line tool that tests and measures AI-written GPU kernels for Apple Silicon models.

Carmen is a command-line tool designed to test, measure, and improve AI-written GPU kernels for Apple Silicon Macs. It uses a loop where Claude writes Metal kernels, and carmen tests them rigorously to keep only the fastest and correct versions. The tool prevents false positives by using hidden test inputs and checking performance against MLX's automatic fusion and real models like Qwen. It is built for developers working with local machine learning models on Apple hardware who want to verify actual speed improvements. Unlike basic benchmarks that accept single-input tests, carmen tests its own tests with planted bugs and measures real-world token speed.

Carmen at a glance

Best for: Software engineer
Runs on: macOS, CLI, open source
Pricing: Free
Released: October 2026
Category: AI coding tool
Website: github.com/utsav1033/carmen

Key features

Tests AI-written Metal GPU kernels
Measures performance impact on real models
Runs rigorous correctness checks with hidden test inputs
Compares stock MLX against custom fused kernels
Measures speculative decoding and KV cache techniques
Profiles model execution step by step

Use it for

Testing custom GPU kernels for correctness
Measuring local model speedups on Apple Silicon
Running speculative decoding benchmarks
Profiling Apple Silicon model execution steps

Alternatives to Carmen

Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
MTPLX — It runs local models twice as fast via native MTP speculative decoding.
Union.ai — Runs machine learning tasks natively on Apple Silicon GPUs using Metal.
llmrun — Check which open models run on your specific GPU or Mac instantly.
GPUStack — Unifies AI model serving and GPU compute management across heterogeneous hardware.
Strata Mlx — Runs 125B MoE models on Apple Silicon by streaming experts directly from disk.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
Muse Wonder — Explore performance benchmarks for Muse Code and Muse Spark.
OptiQ — Runs entirely on Apple Silicon without PyTorch or cloud dependence.
iwant — Spins up cloud GPUs and open models instantly via a single command.
whichllm — Ranks models using real benchmarks and hardware fit instead of parameter count.

All Carmen alternatives