Vlmhub

Vlmhub is a lightweight config-driven framework for unified vision-language model inference across local and cloud backends.

Vlmhub is a lightweight, config-driven framework designed for unified vision-language model inference across both local and cloud backends. It allows users to run multimodal prompts, which include interleaved text and images, against providers like Ollama, MLX-VLM, vLLM, HuggingFace Transformers, Gemini, Vertex AI, OpenAI, or Anthropic using a single interface. The framework relies on a centralized model registry stored in an editable JSON file where models, settings, and generation defaults are managed. Images are automatically encoded as base64 data URIs for API-hosted backends or handled as PIL images for Transformers. It is built for developers who need a consistent way to manage and query diverse vision-language models without altering their underlying inference code.

Vlmhub at a glance

Best for: Software engineer, Researcher
Runs on: Python, Local Server, API
Pricing: Free
Released: October 2026
Category: AI coding tool
Website: github.com/dbiswas55/vlmhub

Key features

Run unified Model.generate calls over text and image blocks
Manage models and settings in a single JSON registry
Support local servers including Ollama, MLX-VLM, and vLLM
Support cloud APIs including Gemini, OpenAI, Anthropic, and OpenRouter
Handle hardware-aware Transformers inference with optional 4-bit quantization
Process interleaved multimodal prompts with automatic image encoding
Manage HuggingFace and Ollama local model caches

Use it for

Running multimodal prompts across different model providers
Managing local and cloud vision-language models through a single config
Testing and grading model outputs with built-in judge scripts
Downloading and cleaning up local HuggingFace and Ollama model caches

Alternatives to Vlmhub

VLMEvalKit — Evaluate large vision-language models across numerous benchmarks with a single command.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
fal — Access 1,000+ generative models and high-performance serverless GPUs.
vLLM — Maximizes throughput with PagedAttention and continuous batching.
ModLens — Converts screenshots and charts into structured JSON data for coding agents.
OpenVLHarness — Lifts six multimodal model backbones by 9.7 to 12.6 points across 22 benchmarks.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
WebLLM — Runs high-performance language models directly inside web browsers.
LightLLM — It delivers high-speed inference and serving performance for large language models.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
MNN — It delivers high-performance inference and training with a lightweight footprint.

All Vlmhub alternatives