Hi everyone, this is Neo.

Three stories from mid-September look unrelated, but they’re all about one thing: the 25-year-old deal of “content in exchange for traffic” has finally been put on the negotiating table.

• September 14: Digiday reported that Google is testing something called the AI contribution pilot inside Search Console — when your content makes a significant contribution to an AI-generated answer, Google pays you.
• September 15: Search Engine Journal contributor Suganthan Mohanadasan published his own experiment — his page quotes a price before an AI agent can read it. Five payments cleared that morning, each with a public transaction hash on-chain.
• July 14: the x402 Foundation formally launched under the Linux Foundation with 40 members, including Visa, Mastercard, Google, AWS, Stripe, Shopify, Coinbase, Cloudflare, and American Express.

A little further back: Cloudflare opened Wallet handle reservations on August 4 and shipped the Monetization Gateway on July 1.

I know that’s a lot of company names. So let me sort the three routes by the three questions that actually matter — where does the money come from, what can you see, and can you use it today — then finish with a preparation checklist.

The three routes, side by side

Route	Who runs it	Where the money comes from	What you can see	Usable today?
Platform revenue share	Google’s AI contribution pilot	Google pays monthly	A single monthly figure, no breakdown	Invited sites only, early pilot
Paid access (B2B)	Cloudflare Pay Per Crawl → Pay Per Use	Crawlers pay for access	Requests and payments	Pay Per Crawl still in closed beta; Pay Per Use in pilot
Your own price tag	Self-built on x402	Agents pay per HTTP request	Everything on your own server	You can test it today (example below)

There’s a fourth route: just block them. Cheapest to run, zero revenue — what you buy is control, not income. More on that later.

Route one: Google pays, but you don’t set the price

The details decide what this actually is.

Based on the help-document screenshots Digiday obtained, plus Barry Schwartz’s writeup, the rules look like this:

• Only contributions during generation count. In Google’s words, AI contribution focuses on the phase where web content can significantly influence what response gets generated. Content that merely confirms facts or gets linked after the answer exists does not qualify.
• Three products are covered: the Gemini app, AI Overviews, and AI Mode. Note that’s one more than Search Console’s generative AI performance report, which only counts impressions from AI Overviews and AI Mode.
• The panel shows one number. Monthly earnings plus some history. Earnings and payments are separate processes, and what actually lands may differ because of tax deductions or local payment thresholds. How the number was calculated is not shown.
• You can opt out any time from Search Console settings.

One publishing executive with knowledge of the program told Digiday it’s “quite black box.” Another source said early returns have been “peanuts” next to ad revenue.

Two things that are easy to miss.

First, the barrier to entry is lower than you’d expect. Google’s previous paid programs were one-off deals with news publishers. This one runs through a panel inside Search Console, and it has already expanded well beyond news sites. Digiday notes small and mid-sized publishers are more interested than large ones.

Second, accepting it has a cost. One publishing executive’s warning deserves to be quoted in full for anyone running an independent site: joining the pilot may weaken your hand in negotiating better terms later, because Google can point to the pilot as payment it already makes.

Neo’s take: this route isn’t pricing — it’s a substitute for copyright compensation. Something that ought to be negotiated by the market becomes a number the platform hands you, from a ledger you can’t see. If you’re a small site with limited leverage, this may be the only cash you’ll ever see from AI answers. But be clear about what you’re trading: not just your content, but your right to ask for more later.

Route two: Cloudflare puts the price inside the request

Cloudflare’s approach is more engineering than policy, and closer to an actual market.

Step one: Pay Per Crawl (July 1, 2025)

The premise was simple: content owners had two options — leave the front door wide open, or build a walled garden. Cloudflare wanted a third path: charge for access. To do it, they revived a status code the web had forgotten:

HTTP 402 Payment Required. It’s been in the spec since 1997, reserved for future use. Twenty-nine years later, it has one.

Domain owners got three settings per crawler: Allow / Charge / Block. One clever detail: you can choose Charge for a crawler that has no billing relationship with Cloudflare at all. Functionally that’s a network-level block, except it tells the crawler a relationship could exist in the future.

Another detail matters for anyone who’s ever been scraped by a fake bot: how do you stop someone spoofing GPTBot? The answer is Web Bot Auth — crawlers generate an Ed25519 key pair, publish the public key in a hosted directory, and sign every request with HTTP Message Signatures. A signature is harder to fake than a user agent, because a user agent can be copied by anyone.

Step two: toward Pay Per Use (2026)

In 2026, Cloudflare shifted its default thinking from “pay per crawl” to “pay per use”, piloting with Ceramic.ai and You.com. The reasoning is blunt: fetching a page doesn’t prove the content was used.

That distinction carries more weight than it looks. “Downloaded” and “used” are very different levels of evidence — and whether your page influenced an answer is something you cannot independently observe. Once your page leaves your server, you have no visibility into how it was used.

Which means the tradeoff of Pay Per Use is this: you have to trust the platform to tell you what happened and what it was worth.

Step three: Wallets (August 4, 2026)

Cloudflare announced Wallets during Agents Week, split into two types:

• Account Wallets belong to humans. You add funds, delegate spend to agents, withdraw when you want.
• Virtual Wallets belong to agents. They operate via API keys, and the maximum spend is capped by the Account Wallet owner. You can configure allowance, allow list, and maximum transaction size.

Their framing of why caps matter is worth stealing: limits look like constraints, but they actually give agents more freedom. If an agent is responsible for $10, you’ll let it explore; if it’s holding $1,000, you’ll watch it.

But be clear about the current state: as of September 15, reserving a handle gets you a page with your name on it and a notification when Wallets goes live. Cloudflare’s documentation explicitly says a reserved handle cannot yet send, receive, or hold funds. The Monetization Gateway is still behind a waitlist.

Route three: x402 — the price tag lives in the HTTP response

The SEJ experiment used the open x402 protocol (the same one Cloudflare’s Monetization Gateway will use), which is why he could ship it without anyone’s approval.

The whole exchange is four steps:

1. The agent requests a page → the server returns 402 Payment Required with a PAYMENT-REQUIRED header describing price and payment requirements;
2. The agent signs an authorization to transfer the exact amount in USDC → retries the request with a PAYMENT-SIGNATURE header;
3. A facilitator verifies the signature and settles on-chain (he used Coinbase’s facilitator);
4. The server returns the content plus a PAYMENT-RESPONSE header containing the settlement receipt.

The offer his server actually sends, decoded from a live 402 response, looks like this:

{
  "x402Version": 2,
  "accepts": [{
    "scheme": "exact",
    "network": "eip155:84532",
    "amount": "10000",
    "asset": "0x036CbD53842c5426634e7929541eC2318f3dCF7e",
    "payTo": "0xEdF2444D0259BBB8aC5094216D0148938F8308ff",
    "maxTimeoutSeconds": 300,
    "extra": { "name": "USDC", "version": "2" }
  }]
}

In plain English: one cent (USDC has six decimals, so 10000 = 0.01), paid to a specific wallet, valid for 300 seconds. A compatible client can read that offer and pay without creating an account and without asking for an API key.

Hit a protected URL with curl and this is what comes back:

$ curl -i https://your-domain.com/protected-page/
HTTP/2 402
payment-required: eyJ4ND...b3Ii...

He also built two layers of spending guardrails for the buying agent — probably the most copyable part of the whole experiment:

• Maximum per call: $0.05;
• Daily allowance: $0.25, tracked in a local ledger.

Exceed either and the script refuses and says why:

[step 3] Spending guardrails
[refused] Price $0.0100 exceeds the per call cap of $0.0010.
This agent does not negotiate. Raise MAX_PER_CALL to override.

Why does that matter more than the price tag itself? Because he surfaces a risk almost nobody discusses: if an agent pays whatever it’s quoted, a paywall stops being a paywall and becomes a toll booth the crawler ignores. You set a price; the buyer never looks at it.

He also built a mode that isn’t enabled in the demo: charge only requests identifying as known AI crawlers (GPTBot, ClaudeBot, PerplexityBot and friends), while human visitors browse free. He tested it with spoofed user agents and it behaved as expected — but he’s honest about the limit: a user agent can be faked. That demonstrates behavior, it doesn’t prove identity. Real verification needs the cryptographic signatures Cloudflare is building.

The single most important line: charging now means giving up citations

I want to pull this out on its own, because it’s the reality every “charge the crawlers” plan runs into.

His own conclusion: GPTBot, ClaudeBot and Googlebot won’t pay your price today. Charge now, and what you lose is citations.

He’s equally clear about what the test was: every payment came from his own agents — no search crawler has ever turned up and paid — and the payments used testnet USDC, which has no real value.

That’s not a technical problem. It’s a market-stage problem:

• The buy side isn’t ready. On Cloudflare’s paid side, bot operators need Web Bot Auth verification, Stripe onboarding, and program approval. An individual agent can’t just join.
• The sell side pays immediately. The moment you charge, you drop out of the citation pool — and right now, citations are the scarcest resource in search.

So the right posture today isn’t charging. It’s preparing.

Five things independent site owners should do now

1. Measure your crawl-to-referral ratio

Open your server logs (or CDN logs), group by user agent, and compute two numbers:

• How many fetches each AI crawler makes per month;
• How much identifiable traffic it sends back.

If something fetches your site 100,000 times and sends you 12 visits, that ratio is where your future negotiation starts. Without that number, every conversation about charging is just a feeling.

2. Stop blocking with a sledgehammer. Classify first.

Type	Examples	How to treat it
Training crawlers	Large-scale corpus collection	Fair game to restrict: no citations, real bandwidth cost
Retrieval/citation crawlers	The ones that let AI cite you	Don’t block these. This is your exposure
User-triggered agents	A person’s agent fetching your page	They’re a visitor. Treat them like one

3. Run a pricing-eligibility audit

Anyone can hang an x402 price tag. Whether money actually arrives depends on how replaceable you are.

Ask yourself: is there anything on this page that exists nowhere else? Original data, first-hand testing, exclusive interviews, verifiable spec or price tables — that’s leverage. Generic explainer content, on the other hand, has substitutes everywhere as far as an AI is concerned.

The conclusion is harsh but useful: content can only have a price if it has no substitute.

4. Build the capability before you flip the switch

You don’t need to charge today, but you can lay the pipe:

• Can your CDN or WAF already distinguish crawlers by user agent or signature and return 402?
• Can it verify Web Bot Auth signatures?
• Do paid pages have a human-readable pricing page and a free preview? Skip either and you’ll annoy crawlers and visitors at the same time.
• What guardrails will you set? Per-call cap, daily allowance, allow list.

5. Watch three dates

• When the Monetization Gateway opens — that decides whether you can adopt paid access without writing code;
• When Cloudflare Wallets can actually move money — right now you can reserve a handle and nothing else;
• Whether Google’s AI contribution pilot expands — and whether it ever publishes its calculation method.

Neo’s take: content is turning from a public signal into a tradeable asset

Three layers to this.

One: this is a new SEO metric. For twenty years, content had exactly one way of being measured — rankings and traffic. Now there’s a second column: how much money (or how many citations) each thousand crawls returns. For content-led independent sites, that column is going into the report eventually.

Two: the answer isn’t picking a side, it’s splitting your site in two.

• Asset content that AI has to read — original data, reviews, comparison tables, definitions. This is the stuff with future pricing power, and the work today is making it the only source (stop syndicating copies everywhere);
• Conversion content only humans need — product pages, pricing, returns, support, checkout. The KPI here is conversion rate, not fetch volume.

Mix the two and you get the worst outcome available: your original data attracts the AI, your bandwidth pays the cost, the visitor ends up somewhere else, and you never book a single conversion.

Three: Google’s payout pilot is cash now and leverage later. I get that a small site can’t easily refuse free money. But do one thing before accepting: write down what you’re giving up. If you ever want to license your content to an AI company and the answer is “Google already pays you for contributions,” what’s your reply?

The end

Here’s the detail I can’t stop thinking about: HTTP 402 was written into the spec in 1997 and left empty. In 2026, twenty-nine years later, it finally answers an old question — “what gives you the right to take what I wrote, for free?”

That question never had an answer, because the technology was open and the economics left no choice. Now the technology has one: 402, signatures, and micropayments. All that’s left is who presses the switch first.

My advice: don’t press it yet. But know where it is.

Because on the day mainstream crawlers actually start paying, the sites that get paid will be the ones that were already keeping the books, already splitting content by purpose, and already the only source for the things they know.