LLMxRay

LLMxRay is an open-source local LLM observatory providing real-time token streaming, quality analysis, and performance profiling.

LLMxRay is a local LLM observatory that provides real-time token streaming, quality analysis, performance profiling, and cost tracking for local models. It requires no cloud connectivity, API keys, or cost, running directly on your machine alongside Ollama. It includes tools such as Chat Diagnostics, Cache Lab, Decision Lab, Model Comparison, Surgical Benchmark, and a Protocol Observatory. The platform is designed for developers, researchers, students, and AI team leads who want to inspect model behavior, evaluate response quality, and analyze token performance visually. It supports features like vector embeddings, RAG pipelines with PDF and DOCX ingestion, tool workshops, and code completion playgrounds.

LLMxRay at a glance

Best for: Software engineer, Researcher
Runs on: Web, Self-hosted, Open source
Pricing: Free
Released: October 2026
Category: AI research tool
Website: github.com/lognebudo/llmxray

Key features

Real-time chat with token intelligence and confidence coloring
Response quality gates for repetition, refusal, gibberish, and truncation
Model comparison workbench with side-by-side streaming and diffs
Performance analytics with latency percentiles and usage heatmaps
Cache lab to analyze KV cache reuse and prefill speed
Decision lab for decision models with probabilities and scores
Protocol observatory comparing native, OpenAI, and Anthropic APIs
Embeddings lab and RAG pipeline supporting local documents

Use it for

Debug local LLM prompts and monitor token streaming speed
Compare responses across different models and temperature settings
Analyze KV cache reuse and optimize prompt prefill performance
Build and test local RAG pipelines with document ingestion
Run surgical benchmarks to test model knowledge with logprobs

Alternatives to LLMxRay

LLM Stats — Independent rankings updated continuously from live API metrics.
OpenMed — Runs 100 percent on-device without cloud calls or data sharing.
Decido — Runs locally on CPU while beating published benchmark scores.
Token Barometer — Compare daily list prices and history for thousands of language model offerings.
Overview — Learn language models by running hands-on experiments without writing scripts
VLMEvalKit — Evaluate large vision-language models across numerous benchmarks with a single command.
UltraRAG — Build complex RAG workflows using modular MCP components and YAML configuration.
Vegaml — Runs locally on your hardware with calibrated probabilities and conformal answer sets.
notebooklm-py — Automates research, content generation, and structured data export from notebooks.
OpenCompass — It offers an objective, comprehensive, and credible open-source AI evaluation system.
LightRAG — It uses a dual-layer architecture to reduce LLM calls and retrieval latency.
FlashRAG — It provides pre-processed benchmark datasets and pre-implemented algorithms.

All LLMxRay alternatives