Kyojin

Kyojin is an inference engine that runs large language models on AMD Strix Halo hardware.

Kyojin is an inference engine built on ExLlamaV3 for AMD Strix Halo mini PCs. It supports models around 100 GB in size on 128 GB systems using speculative decoding. The software includes an uncensored mode that reduces model refusals without altering model weights, available as ready-made packs or presets. The server provides an OpenAI API endpoint on port 8000 and includes a Prometheus metrics endpoint. It targets the Ryzen AI Max+ 395 processor with ROCm and targets developers and users running large language models locally on specific hardware.

Kyojin at a glance

Best for: Software engineer
Runs on: Linux, API, self-hosted, open source
Pricing: Free
Released: October 2026
Category: AI coding tool
Website: github.com/yamz-labs/kyojin

Key features

Inference engine built on ExLlamaV3
Speculative decoding support
Uncensored mode for reduced refusals
OpenAI API compatible server on port 8000
Prometheus metrics endpoint
Support for models up to 105 GB

Use it for

Run large language models locally on AMD hardware
Serve models using an OpenAI compatible API
Monitor server performance with Prometheus metrics

Alternatives to Kyojin

oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.
Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
Infernix — Serves massive Qwen models quickly on a single consumer GPU with advanced caching.
vLLM — Maximizes throughput with PagedAttention and continuous batching.
Resin — Builds high-performance inference engines for complex models like DeepSeek and GLM.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
FreeToken — Runs 290B+ MoE models on consumer gaming PCs using elastic edge resources.
Shimmy — Runs local AI models with zero Python or C++ dependencies.
Xinference — Supports a wide range of model types and hardware, including multi-node setups.
Lumabri — Runs large AI models across multiple computers by sharing memory and layer workloads.

All Kyojin alternatives