Pith

Pith is an open-source optimization tool that cuts LLM output token spend without losing information.

Pith is an open-source tool that cuts LLM output-token spend without losing information. It watches a route, records usage and sample bodies into a SQLite file, replays samples under cheaper output profiles, and uses a judge model to compare answers with the baseline. It pins a profile only when it matches the baseline quality and its savings repay the sweep within a month. It never touches system prompts or tools so prompt caches keep hitting, and it fails open on any error. It runs as a standalone proxy or integrates with LiteLLM, Headroom, and Portkey.

Pith at a glance

Best for: Software engineer
Runs on: CLI, self-hosted, open source
Pricing: Free
Released: October 2026
Category: AI coding tool
Website: github.com/dex0shubham/pith

Key features

Watches routes and replays samples under cheaper output profiles
Uses a judge model to compare candidate answers with the baseline
Pins profiles only when quality matches and savings repay the sweep
Stores usage and request samples in a SQLite file
Integrates with LiteLLM, Headroom, and Portkey
Supports request-level and global kill switches

Use it for

Cutting LLM output-token spend for support tickets and summarisation
Sweeping routes from the command line interface
Integrating optimization into LiteLLM proxy setups

Alternatives to Pith

llm-d — Deploys production-grade LLM serving on existing hardware clusters.
Paritok — It reduces token bills and increases session length by compressing prompt data.
LangChain4j — Integrates LLMs and vector stores directly into Java applications.
Rubi — It cuts AI token waste by keeping sessions short and mapping project knowledge.
PocketFlow — It offers a 100-line minimalist codebase with zero bloat and dependencies.
ilha — Prevents coding interruptions by switching AI accounts before limits are reached.
Mcpspan — Track MCP tool usage, token costs, and errors privately on your own machine.
Opik — It offers comprehensive tracing, automated evaluations, and open-source flexibility.
Codeflash — Automated performance engineering that identifies and implements code optimizations.
SmallCode — Runs fully local with budget-managed context and a forgiving parser for small models.
Headroom — Reduces token usage by 60 to 95 percent for JSON and lowers costs while maintaining the same answers.
Squeeze — Cuts token usage for coding agents by shrinking verbose tool output while retaining key data.

All Pith alternatives