Every number in a tiny GPT

Every number in a tiny GPT is an interactive transformer visualization tool that shows exact calculations for a tiny GPT model.

Every number in a tiny GPT is an educational tool that displays the complete, working architecture of a tiny GPT model. Shrunk from GPT-2 and trained on 36 toy sentences, the model runs live computations directly in the browser. Users can step through inference, pretraining, and post-training phases, including one PPO iteration for RLHF. The interface breaks down every step from tokenisation and embeddings to attention heads and final softmax sampling. It is designed for students, developers, and researchers who want to understand the exact mathematical operations inside transformer models.

Every number in a tiny GPT at a glance

Best for: Software engineer, Researcher
Runs on: Web
Pricing: Free
Released: October 2026
Category: AI research tool
Website: llm.manogya.dev

Key features

Step through inference, pretraining, and post-training
Inspect exact numbers for every calculation
Change the input prompt with live recomputation
View tokenisation and embedding steps
Examine attention scores and causal masks
Trace MLP expansions and GELU activations
Explore final layer norm, logits, and sampling

Use it for

Study transformer model architecture step by step
Visualize matrix multiplications and normalisations
Learn how reinforcement learning from human feedback works
Inspect token embeddings and attention heads

Alternatives to Every number in a tiny GPT

Transformer Explainer — Visualizes internal Transformer operations and next-token prediction in real time.
MiniCPM — Offers 2B-class open-source SOTA performance with compact on-device deployment.
Finetuning With Sampling — Official implementation of the Finetuning with Sampling research paper from Harvard.
Overview — Learn language models by running hands-on experiments without writing scripts
NanoJev — It runs parallel decisions and outputs probability distributions without output-token decoding.
DeepInfer Studio — Explore AI inference workflows and resource trade-offs without requiring expensive hardware or GPUs.
Ncm — Generates text significantly faster than diffusion baselines while maintaining high quality.
OpenVLHarness — Lifts six multimodal model backbones by 9.7 to 12.6 points across 22 benchmarks.
GPT Researcher — Ranked #1 on Carnegie Mellon's DeepResearchGym benchmark for academic research.
WeMM Embedding — Provides state-of-the-art performance across multiple multimodal benchmarks.
FigGenie paper diagram skill — Learned from real top-venue figures with built-in linting and checks.
Decido — Runs locally on CPU while beating published benchmark scores.

All Every number in a tiny GPT alternatives