Pay-per-Crawl

Pay-per-Crawl is a web monetization gateway charges AI crawlers USDC on Base to access website content directly.

ForgeMesh Pay-per-Crawl is a monetization gate that allows website owners to charge AI and SEO crawlers for access to content. Humans and verified search engines read for free, while AI crawlers receive an HTTP 402 challenge and pay in USDC on Base directly to the site owner's wallet. There is no custody, no account required, and no API key needed. It supports ten distinct tech stacks including WordPress, Drupal, Laravel, Next.js, Node, Cloudflare Workers, nginx, Caddy, Traefik, and Python. The system serves discovery files like llms.txt and agents.txt while utilizing a shared agent catalog to identify incoming traffic.

Pay-per-Crawl at a glance

Best for: Software engineer
Runs on: Web, API, Self-hosted
Pricing: Pro $4.99/month for up to 5 sites
Released: October 2026
Category: AI coding tool
Website: forgemesh.io

Key features

Monetize AI crawlers with USDC on Base
Serve discovery files like llms.txt and agents.txt
Integrate with WordPress, Drupal, Laravel, and Python
Support Next.js, Node, and Cloudflare Workers
Provide Caddy, Traefik, and nginx modules
Bypass fees for humans and search engines
Direct payments to owner wallet with no custody

Use it for

Charge AI crawlers for scraping website content
Block or monetize automated SEO scripts
Protect web content from uncompensated AI training

Alternatives to Pay-per-Crawl

CrawlForge MCP — Provides 31 MCP-native web scraping tools with transparent credit pricing.
ClawRouter — Routes requests across 82 models locally in under 1ms to reduce AI API costs.
ForgeCode — Leads TermBench 2.0 and runs natively right inside the terminal.
CrawlRaven MCP — It provides typed Search Console and GA4 data directly inside AI chats.
Scrapling — It uses adaptive element tracking to maintain scrapers when websites change.
Agentfork — It replays steps from cache and runs sweeps to provide pass rates instead of anecdotes.
Magpie — It enables AI agents to use models not natively supported by their internal protocols.
AI Deploy Bench — Measures AI agent success, time, tokens, cost, and human steps for cloud deployments.
QueryForge AI — Enforces application rules and access controls before generating queries.
InsForge — Provides built-in infrastructure services for AI coding agents to ship apps end to end.
Agentic Bug Hunter — It drives the reconnaissance, triage and reporting loop an agent would run.
PullRequest — Combines AI risk analysis with vetted human experts to catch critical vulnerabilities.

All Pay-per-Crawl alternatives