GPUWorks

GPUWorks is a Model-as-a-Service and GPU management platform that deploys and runs AI models across heterogeneous hardware clusters.

GPUWorks is an ultra-lightweight GPU management and Model-as-a-Service platform. The control plane and web UI download as a 40 MB binary, storing state in a single SQLite file. Users join GPU servers, rented cloud instances, or Macs, pick a model, and the platform estimates memory, schedules GPUs, and serves models via an OpenAI-compatible endpoint. It supports NVIDIA, AMD, Huawei Ascend, T-Head PPU, Hygon DCU, and Apple Silicon side by side. The built-in catalog contains 1,235 launch specs for 392 models including chat, vision, embedding, speech, and video models. It handles multi-node parallelism, model sharing on single GPUs, sleep/wake cycles, and prefill/decode disaggregation.

GPUWorks at a glance

Best for: Software engineer
Runs on: Linux, macOS, API, self-hosted
Pricing: Free
Released: November 2026
Category: AI coding tool
Website: github.com/flitai-io/gpuworks

Key features

Deploys verified model and accelerator combinations in one click
Schedules heterogeneous hardware in a single cluster
Shares GPUs between models with hard memory quotas
Provides an OpenAI-compatible gateway with token usage tracking
Estimates memory and explains placement before startup
Monitors GPU metrics, searchable logs, and per-request traces
Offers a built-in benchmark that sweeps concurrency to find SLA limits

Use it for

Serving chat, vision, embedding, speech, and video models
Managing GPU clusters across public and private clouds
Running fine-tuning and training jobs on accelerators
Sharing a single GPU among multiple AI models

Alternatives to GPUWorks

GPUStack — Unifies AI model serving and GPU compute management across heterogeneous hardware.
Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
iwant — Spins up cloud GPUs and open models instantly via a single command.
Narwhal — It automatically hot-swaps prefill and decode roles without reloading model weights.
Union.ai — Runs machine learning tasks natively on Apple Silicon GPUs using Metal.
llmrun — Check which open models run on your specific GPU or Mac instantly.
fal — Access 1,000+ generative models and high-performance serverless GPUs.
Imp — Runs local models directly on NVIDIA Blackwell hardware using native 4-bit math.
Replicate — Run and fine-tune thousands of open-source and custom AI models via a single cloud API.
Reflex — It is optimized for cold-start latency and single-tenant decisions.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.

All GPUWorks alternatives