whereismyai
Backburner
Backburner: This open source llama.cpp fork enables a connected iPhone to accelerate large model inference and extend context capacity on a Mac over USB-C.
Backburner distributes large language model computation between a Mac and a connected iPhone over a 10 Gb/s USB-C cable. It modifies the llama.cpp engine to split inference layers, running the first 40 layers on the Mac and the remaining layers on the iPhone GPU. This configuration accelerates prefill speeds for large context windows and allows the iPhone to store and compute attention over oldest key-value pairs beyond the Mac's local memory limits. The system utilizes the iPhone's Neural Engine and GPU for attention calculations, maintaining token-identical outputs compared to Mac-only execution. It is designed for developers and power users working with models like Qwen3.8-27B who require extended context retention and faster prompt processing.
Backburner at a glance
Key features
- Split prefill architecture for distributed layer processing
- Offloading of oldest context keys to iPhone storage
- iPhone Neural Engine utilization for attention calculations
- Mac CPU SME2 acceleration for matrix multiplication
- DFlash2 speculative decoding with recurrent-state replay
- SSD-based prompt caching for faster session restoration
- USB-C hardware connectivity between Mac and iPhone
Use it for
- Extending context windows for long-running agent sessions
- Reducing latency during large file or tool result processing
- Running larger LLM context sizes than local Mac memory allows
Alternatives to Backburner
- AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
- oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.
- LMDeploy — Accelerates inference with high-performance engines and quantization tools.
- llm-d — Deploys production-grade LLM serving on existing hardware clusters.
- MTPLX — It runs local models twice as fast via native MTP speculative decoding.
- Lithos Metal — It delivers low-latency LLM inference and local serving on Apple silicon.
- Distributed Llama — Splits model memory and computation across multiple connected home devices.
- Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
- Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
- Kapitsa — Optimized inference engine delivering high token generation speeds on Apple Silicon.
- MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
- Pulsar — It is the fastest inference engine for Qwen models on Apple Silicon.
All Backburner alternatives