Strata Mlx

Strata Mlx is an MLX inference engine that runs Qwen3.8-Flash-Next on Apple Silicon using Strata GGUF files and line protocol.

Strata Mlx is an independent, experimental MLX inference engine designed to execute the 125B-parameter Qwen3.8-Flash-Next model on Apple Silicon Macs. It reads Strata's GSQ-RCO GGUF files (such as Q20 and IQ3S) directly without altering weights, offloading unused experts to the SSD when unified memory is insufficient. The engine features speculative decoding via the model's draft head, layer-by-layer prompt processing, and Metal matrix kernels. It communicates through Strata's line protocol over standard input and output, allowing integration with Strata's existing server and web UI.

Strata Mlx at a glance

Best for: Software engineer
Runs on: macOS, CLI, open source
Pricing: Free
Released: October 2026
Category: AI coding tool
Website: github.com/yibie/strata-mlx

Key features

Loads GSQ-RCO GGUF files directly without modifying weights
Streams mixture-of-experts parameters from SSD when out of memory
Uses model draft head for speculative draft-and-verify decoding
Speaks Strata line protocol over standard input and output
Caches conversations in memory with --conversation-cache-mib
Processes long prompts layer by layer to reduce memory usage
Maintains GPU responsiveness with a background keep-awake mode
Assesses Mac hardware suitability through the --check command

Use it for

Running 125B mixture-of-experts models locally on Apple Silicon hardware
Serving Qwen3.8-Flash-Next to Strata web frontends and servers
Managing long-context conversations measuring up to 128K tokens

Alternatives to Strata Mlx

MTPLX — It runs local models twice as fast via native MTP speculative decoding.
Rapid-MLX — Run 250 MLX-native models locally on Apple Silicon with high inference speeds.
MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
fal — Access 1,000+ generative models and high-performance serverless GPUs.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
Kapitsa — Optimized inference engine delivering high token generation speeds on Apple Silicon.
OptiQ — Runs entirely on Apple Silicon without PyTorch or cloud dependence.
Infernix — Serves massive Qwen models quickly on a single consumer GPU with advanced caching.

All Strata Mlx alternatives