Imp

Imp is an LLM server for NVIDIA Blackwell GPUs that runs local models for chat apps and coding agents.

Imp is a dedicated large language model server built exclusively for NVIDIA Blackwell GPUs, including every GeForce RTX 50 and RTX PRO card. It runs local models and serves them to chat applications and coding agents through OpenAI and Anthropic compatible APIs. The software utilizes native 4-bit math, tensor cores, and memory bandwidth of a single Blackwell card without needing a CUDA toolkit or Python on the host machine. It supports GGUF files and SafeTensors in NVFP4, handles mixture-of-experts models larger than video memory by placing experts in host RAM, and includes built-in chat, continuous batching, and prefix caching. The project is distributed under the MIT license.

Imp at a glance

Best for: Software engineer
Runs on: Linux, Windows, API, self-hosted, open source
Pricing: Free and open source under the MIT license
Released: October 2026
Category: AI coding tool
Website: github.com/kekzl/imp

Key features

Serves LLMs via OpenAI and Anthropic compatible APIs
Supports GGUF files and SafeTensors in NVFP4
Handles models larger than VRAM using host RAM
Provides built-in chat interface
Implements continuous batching for multiple users
Caches repeated prompt prefixes

Use it for

Run local language models on NVIDIA Blackwell GPUs
Point coding agents at a local inference server
Execute mixture-of-experts models exceeding VRAM capacity

Alternatives to Imp

Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
GPUStack — Unifies AI model serving and GPU compute management across heterogeneous hardware.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
iwant — Spins up cloud GPUs and open models instantly via a single command.
MTPLX — It runs local models twice as fast via native MTP speculative decoding.
whichllm — Ranks models using real benchmarks and hardware fit instead of parameter count.
OpenCode — Works with 75+ model providers, local models, and existing subscriptions
fal — Access 1,000+ generative models and high-performance serverless GPUs.
Infernix — Serves massive Qwen models quickly on a single consumer GPU with advanced caching.
Project Maya — Runs a 321B-parameter AI model locally across VRAM, RAM, and NVMe SSD.
Shimmy — Runs local AI models with zero Python or C++ dependencies.

All Imp alternatives