whereismyai
Turbo Fieldfare
Turbo Fieldfare: Runtime for Gemma 4 26B-A4B inference that operates within 2 GB of RAM on Apple Silicon Macs by streaming model experts from storage.
Turbo Fieldfare is a Swift and Metal-based runtime designed for Gemma 4 26B-A4B inference. It reduces memory requirements by keeping only the core and KV cache in RAM while streaming necessary model experts from SSD on demand. This approach allows large language models to run on Apple Silicon Macs with as little as 8 GB of memory. The package includes a native Mac app for chat, a command-line interface, and an OpenAI-compatible loopback server. Users can optionally install a companion vision tower to enable image support.
Turbo Fieldfare at a glance
Key features
- Native Mac application for local generation
- Command-line interface for chat and completion
- OpenAI-compatible loopback server
- Streaming model installer
- Support for vision models via companion pack
- Metal-accelerated inference runtime
Use it for
- Running 26B parameter models on 8 GB RAM machines
- Integrating local inference into macOS workflows
- Automating completions via command-line interface
- Developing applications using OpenAI-compatible API endpoints
Alternatives to Turbo Fieldfare
- Ane Fieldfare — Runs Gemma 4 26B-A4B at over 100 tokens per second on a 16 GB Mac.
- oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.
- LMDeploy — Accelerates inference with high-performance engines and quantization tools.
- AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
- Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
- Backburner — Extends LLM context limits and speeds up prompt reading using iPhone hardware.
- Strata Mlx — Runs 125B MoE models on Apple Silicon by streaming experts directly from disk.
- Kapitsa — Optimized inference engine delivering high token generation speeds on Apple Silicon.
- Pulsar — It is the fastest inference engine for Qwen models on Apple Silicon.
- MTPLX — It runs local models twice as fast via native MTP speculative decoding.
- ms-swift — Supports hundreds of text and multimodal models with a full training pipeline.
- Rapid-MLX — Run 250 MLX-native models locally on Apple Silicon with high inference speeds.
All Turbo Fieldfare alternatives