whereismyai
Self Bench
Self Bench is a benchmarking platform that tests coding agents and models against an organization's own merged pull requests and hidden tests.
Self Bench converts a repository's merged pull requests into executable benchmark tasks. The system utilizes an authoring agent to generate instructions, hidden tests, and reference solutions from existing commit history. Users manage tasks, approve benchmarks, and execute coding models or agents within secure sandboxes. Results plot accuracy against cost to provide performance data based on proprietary codebases. This tool supports public leaderboards for open-source projects and includes a self-hosted deployment option on Google Cloud Platform. It targets development teams aiming to quantify how specific models perform on unique internal code patterns.
Self Bench at a glance
- Best for: Software engineer
- Runs on: Web, self-hosted
- Pricing: Free
- Released: August 2026
- Category: AI coding tool
- Website: github.com/mupt-ai/self-bench
Key features
- Benchmark generation from merged pull requests
- Automated creation of hidden tests and reference solutions
- Accuracy and cost performance analysis
- Visual leaderboards for model comparison
- Task approval and filtering workflows
- Self-hosted deployment on GCP with Terraform
- Integration with coding models and sandboxes
Use it for
- Evaluating LLM performance on proprietary code
- Comparing coding agent efficiency across different model settings
- Validating agent accuracy using repository-specific unit tests
- Building public benchmarks for open-source projects
Alternatives to Self Bench
- Rigmark — Measures local AI performance with reproducible, agent-like workloads.
- AI Deploy Bench — Measures AI agent success, time, tokens, cost, and human steps for cloud deployments.
- jevgrep — Reduces coding agent costs by providing targeted, relevant repository context.
- BenchGen — It enables stress-testing AI agents in safe, simulated operational environments.
- pstack — Improves AI coding agent outcomes with rigorous workflows.
- OpenCode — Works with 75+ model providers, local models, and existing subscriptions
- Skill Placebo — Compares coding agent skills against neutral text placebos to measure actual impact.
- aiXcoder — Provides full-stack private deployment for secure, enterprise-wide code generation.
- Feedback Bench by Enterpret — Ranks 17 coding agents using public user feedback on 63 criteria.
- Agent QA — It uses repository-based memory to learn from test runs and perform self-healing.
- Codeflash — Automated performance engineering that identifies and implements code optimizations.
- Autoloom — It embeds engineering judgment, change necessity checks, and evidence-backed delivery into automated coding.
All Self Bench alternatives