whereismyai
EvalCart
EvalCart is an evaluation platform that tests grocery-shopping agents against fixed catalogs, household constraints, and budgets.
EvalCart provides a testing environment for grocery-shopping agents using a frozen grocery catalog, a deterministic cart scorer, and a fake store for agents and people. It tests agents against real shopping constraints like household needs, pantry items, leftovers, and budgets. The platform is designed for developers testing shopping agents, product managers tracking performance, and researchers comparing agent implementations. It features a public benchmark, an agent leaderboard, and tools to build custom shopping tasks. The system uses a synthetic product catalog and does not connect to grocery retailers or place real orders.
EvalCart at a glance
- Best for: Software engineer
- Runs on: Web, API
- Pricing: Free
- Released: November 2026
- Category: AI research tool
- Website: evalcart.com
Key features
- Test shopping agents against fixed catalogs
- Score carts with a deterministic scorer
- Compare submitted agents on a leaderboard
- Build custom shopping tasks and budgets
- Score carts through the API
- Run tasks in the test store
Use it for
- Test grocery-shopping agents before and after code changes
- Compare agent implementations on a public leaderboard
- Build and save custom shopping tasks
- Score agent carts against budgets and constraints
Alternatives to EvalCart
- Eval Skills — Guides coding agents from first execution to validated graders and experiments.
- OpenResearch — Run parallel research workflows locally or on remote compute.
- EvalScope — It combines model capability benchmarks with inference performance testing in one tool.
- Paper2Agent — It enables direct interaction with research papers and automatic result reproduction.
- Rsi Eval — It supplies the evaluation boundary so methods can use existing benchmarks without reimplementing execution.
- LemMeBuyIt — Provides live pricing and product matching across over 100 U.S. retailers for shoppers and AI agents.
- ScopeWeaver — Coordinates autonomous security agents and asset scope in a single console.
- LMMs-Eval — Provides open research and tooling for multimodal model evaluation and benchmarks.
- Senro — Tests WebMCP tools and monitors AI agent behavior without building infrastructure.
- Navital Gym — Measure AI agent performance on realistic tasks using deterministic rules without LLM judges.
- SoL Pi — Saves $8.75 to $13.50 per hour compared to native Codex and Claude Code systems.
- ktx — It gives data agents a shared context workspace of approved metrics and git-based files.
All EvalCart alternatives