Charging AI Bots: Revenue vs Visibility

Charge or let them pass? Site owners now face a new choice: bill AI crawlers for access, or let them in free for the sake of being cited in AI answers. There's no clean answer — and that's the whole debate.

The case for charging. For thirty years crawlers paid in traffic: they indexed your content and sent visitors back. AI bots increasingly take the text and return nothing. The industry's answer is paid access via the HTTP 402 status. Cloudflare (July 2025) and AWS (June 2026) launched pay-per-crawl at the edge, joined by TollBit and Akamai — per-request billing runs on the x402 protocol.

The case against. The bot you'd invoice and the bot that sends you a customer from an AI answer are often the same pipeline. Per Adobe, AI-referred traffic to retailers jumped 393% year over year. Block access and you can easily vanish from the answers already bringing buyers. The full breakdown is on Search Engine Journal.

Who should do what:

• Charging makes sense for news outlets, reference databases and archives with independent distribution;
• Not risking it suits sites and e-commerce that live on visibility inside AI answers.

Our take: this is a visibility decision, not a quick-money one. Don't cut access blindly. First meter individual bots on specific sections, measure the real load and the citation impact — and only then decide who gets in free and who pays.