Hi everyone, this is Neo.

Today I want to talk about something subtle: why GEO (Generative Engine Optimization) tools aren’t selling.

You’ve probably noticed the flood of SaaS tools over the past year claiming to “measure AI visibility” — tracking how often your brand gets mentioned or cited in ChatGPT, Perplexity, Google AI Overviews, what position you hold, and so on. Prices run from a few hundred to a few thousand dollars a month, and the pitch decks look stunning.

But a recent survey from Duane Forrester, a former Bing executive, pulled back the curtain on the industry: the data is valuable. The tools, nobody trusts.

This isn’t my gut feeling — it’s the honest vote of 163 practitioners on the front lines. Let me break down what this survey actually found, and what it means for independent site owners deciding whether to buy these tools.

The two numbers that tell the whole story

First, some background. In July, Duane Forrester (who spent years at Microsoft Bing and now runs an AI search consultancy) ran a three-week survey aimed at people who make a living from AI search visibility — SEO consultants, agencies, and in-house teams — asking what they think of the platforms built to measure it.

He got 163 valid responses. The sample was recruited through his own network, two newsletter pushes, two respected industry newsletter mentions, plus paid ads on LinkedIn and X. So it describes engaged practitioners in one corner of the industry — not the whole industry. Keep that caveat in mind; it becomes part of the story later.

Two sets of questions:

1. How valuable is this kind of AI visibility data? (covering query alignment beyond keywords, competitor comparison, whether a mention comes from training or retrieval, chunk-level attribution, citation status)
2. Is investing budget in a dedicated measurement platform worthwhile right now?

The results:

Question	Score (out of 5)
Value of AI visibility data	4.20
Willingness to invest in a platform	3.19

Only 44% think buying a tool in this category is worthwhile at all, and nearly a third rate it 1 or 2 out of 5.

Data worth 4.20, tools worth 3.19 — that one-point gap between them is the most valuable finding in this report.

And it’s not sampling noise. Forrester collected the data in four snapshots (36 → 75 → 100 → 163 responses), and neither number moved more than a tenth of a point the entire run. “Whatever this is, it isn’t a sampling artifact. It settled early and stayed put while the sample quadrupled.”

Neo’s take: This gap feels painfully real. When GEO tools first got hot, I watched plenty of independent site owners in my circle impulse-buy annual plans because “how can you survive the AI era without measuring brand visibility?” Then the cold shower: how is this number actually computed? Why should I trust it? That’s exactly the question most sellers never ask before checking out. 4.20 vs 3.19 isn’t one person’s bias — it’s the whole industry hesitating in unison.

Why nobody’s buying: 57% say “I don’t trust the number”

Here’s the most striking comparison in the survey:

• Only 7% complained about price
• 57% said either “I don’t believe the number” or “I can’t connect this to money”

Price is not the objection. Trust is.

75% of respondents (123 people) wrote something in the open text box. Forrester expected a feature-request list. That’s not what he got. Categorized, the themes look like this:

Theme	Share of responses
Trust, accuracy, opaque methodology	24%
ROI and attribution to business value	20%
Non-determinism, variance, personalization	15%
Synthetic prompts vs real user demand	11%
Not actionable: “what do I do now”	9%
Price or cost	7%
Citation vs mention vs recommendation	4%
Attribution to a specific passage	4%
Prompt-count limits	4%
“No GSC equivalent for LLMs”	3%

A few of the open-ended answers sting. Let me share the good ones:

1. “Prompt-list tracking is a self-fulfilling prophecy.” The standard play: you pick a list of prompts, then the tool measures your visibility against them. But here’s the problem — you chose the prompts, which means you decided in advance where you should be visible, then measure yourself against your own list. One respondent called it exactly that: a self-fulfilling prophecy.

2. “These tools have no denominator.” A rank tracker has a clear denominator — your position among everyone ranked. But AI visibility scores come from invented prompt lists rather than observed query volume. Ordinary model variance gets reported to a client as a win or a loss, with nothing underneath to say which.

3. “Citation tools are a different animal from rank trackers.” You can’t reverse-engineer what’s working when the answer changes every time you ask, one respondent argued. What you’re left with is closer to a brand awareness signal than a diagnostic.

4. An agency running 50-plus clients spelled out the squeeze: Without measurement tools, they can’t sell “AI visibility” as a service. And they can’t justify the tools until they’re selling the service. A dead loop.

5. The sharpest one, aimed straight at vendors:

“You can’t do what these tools purport to do, because every user of every model gets a different experience. Snake oil. Magic beans.”

Neo’s take: Write this down: “no denominator.” As independent site owners, we’re used to Search Console showing impressions and CTR — numbers backed by real Google user behavior. But many GEO tool scores come from asking an AI model a few prompts, counting how many times your brand shows up, then dividing by a denominator they invented themselves. When you control the denominator, you control the score. That’s not to say every GEO tool is a scam — it’s to say: figure out where the number comes from before you pay for it.

The cross-check: it’s not a pricing problem, it’s a belief problem

Forrester cross-referenced the responses and found three patterns that held up:

Pattern one: people who doubt trust value the data exactly as much as everyone else.

• Trust-skeptics’ rating of data value: 4.20
• Everyone else’s: 4.19

A hundredth of a point apart. Budgets statistically indistinguishable.

But their willingness to invest in a platform drops to 2.76 vs 3.36 for everyone else, and they’re markedly less likely to be paying for anything.

Same valuation. Same money available. Different conclusion. What’s blocking this segment isn’t what the data is worth, and isn’t what they can afford. It’s whether they believe it.

Pattern two: almost nobody is building the alternative.

Only 8% of respondents built their own tooling. Among people who raised trust or non-determinism concerns: 9%. Among the 78% who call accuracy essential in a vendor: 6%.

Forrester doesn’t read this as hypocrisy — building this stuff is genuinely hard — but it reframes the objection. “The numbers can’t be trusted” isn’t functioning as a diagnosis anyone acts on. It’s a request for someone else to solve it properly.

Pattern three: the only thing that closes the gap is having bought.

The gap between valuing the data and funding a platform is identical across every role — agencies, in-house teams, independent consultants are all within a rounding error of each other.

Except by subscription status:

• Among current subscribers, the gap nearly disappears
• Among everyone else, it’s three times larger

Forrester honestly admits he can’t tell which direction that runs — buying may resolve the doubt, or people without doubt may be the ones who buy. But it’s the single largest split in the dataset. And there’s a suggestive hint: subscribers were more likely to say “the data doesn’t tell me what to do” and less likely to say “I doubt the data.” People who haven’t bought doubt the numbers. People who have bought accept the numbers — and can’t act on them.

Neo’s take: Pattern three is worth chewing on. “Buyers stop doubting” — in marketing terms, there’s an uglier name for that: cognitive dissonance (you spent the money, so you convince yourself it was worth it). But it could also genuinely be that you have to use the thing before you understand its limits. My advice to independent site owners: if you really want to try one, start with the monthly plan or the smallest tier — treat it as a field trip, not a marriage. Run it for three months, compare its scores against your real business data, verify it yourself, and then decide on renewal.

The vendor’s dilemma: methodology is the asset — publishing it is suicide

The most-raised objection in the survey was methodology opacity: “Show me where this data comes from and why I should believe it.”

Fair ask. Also, as Forrester (who built one of these platforms himself — he’s upfront about the conflict of interest) puts it bluntly: no vendor in this category can give it to you.

The reason is brutal:

For a venture-backed company, the methodology is the asset. Publishing it converts the product into a free tool with a burn rate and a board. Any vendor who appears to have opened the box has shown you a curated subset. The disclosure you asked for is either commercially fatal or theatre — there is no third option.

And this is not unique to GEO tools. It’s true of every measurement business that has ever existed, including the keyword tools this industry has trusted for twenty years without ever seeing inside them. Forrester worked inside one of those systems for almost a decade, so he’s not guessing.

But the question still hangs: if you can’t have the methodology, what would actually make you believe a number? Reproducibility? Published variance? Third-party audit? None of the 163 respondents proposed one. That gap deserves more attention than it’s getting.

Neo’s take: Read that paragraph twice. It’s not defending GEO tools — it’s telling us a hard industry truth: measurement tools’ opacity isn’t a moral failing of these companies, it’s a business-model requirement. Independent site owners should recognize the pattern — Ahrefs and Semrush never published their ranking methodology either, and we’ve lived with that for years. The difference: Google rankings are public output that third parties can independently verify. AI model answers are a black box, and there’s nowhere to verify from. A tool’s black box stacked on a model’s black box — that’s a double black box. Get that straight before you buy.

The deeper problem: this may not be measurable at all

Here’s the argument that really scares vendors — and Forrester admits he can’t dismiss it:

AI visibility may be unmeasurable in principle.

The case:

• AI systems are non-deterministic: ask the same prompt ten times, get ten different answers;
• Every user’s experience is personalized: what you see isn’t what your neighbor sees;
• So “a snapshot of what a model said on Tuesday to a synthetic prompt” may be measuring nothing that generalizes to anything.

Forrester: “I don’t think that’s right. But I can’t prove it isn’t — and neither can anyone selling you a dashboard.”

Kevin Indig (independent SEO consultant; I did a deep dive on his AI Halftime Report a while back) has made a similar argument:

Tracking the presence of a brand in AI search is severely complex. You need to factor in the engine, personalization, reasoning levels, model updates, stochastic variability, etc. Measurement is fragmented. 91% of citations appear in only one of ChatGPT, Perplexity, or AI Overviews. Prompt tracking should be closer to polling and focus groups than SEO rank tracking.

Neo’s take: Memorize that 91% stat — being cited in ChatGPT says nothing about being cited in Perplexity, and a sudden “visibility drop” in one platform may just be a model update with zero connection to you. Kevin Indig is right: AI visibility measurement is closer to opinion polling than medical testing. Samples, timing, question wording — all of it shapes the result. Polls have margins of error and you accept that. The danger is treating one poll as a diagnostic report. Tool data is a direction, not a verdict.

The final awkward number: 163 people out of 715,000

The report ends with a number Forrester himself circled for a long time before writing it down:

IBISWorld counts roughly 715,000 people employed in SEO and internet marketing consulting in the United States alone. This survey: 163 responses. About one in every 4,400.

And those 163 were hard-won: two newsletter pushes, two respected industry newsletters, amplification from around twenty people in the space, a full week of paid promotion on two platforms. The response trickled.

The kicker: he stopped asking politely and pointed out how few people had bothered — and got more responses in two days than the previous week had in total.

His summary is restrained:

It’s hard to square an industry that describes this as an existential threat with a sample this hard to assemble. The gap between how much this gets talked about and how much of it is actually being done may be the most honest finding here.

The data is worth 4.20/5. The platforms are worth 3.19/5. And 163 people out of 715,000 gave their three minutes to say so.

Neo’s take: That gap between “discussion volume” and “actual practice” — I feel it in my bones. Same in the independent site community: everyone’s talking AEO, GEO, AI search on social media, but how many have actually built a prompt panel? How many are actually tracking brand mentions across AI platforms? The ratio is probably just as grim. And that’s precisely the opportunity: while the crowd is still discussing AI visibility, if you’re already measuring it and optimizing it, you’re ahead. You weren’t among those 163 — fine. Starting today, you can be one of the few thousandths who actually do the work.

Practical advice for independent site owners

Enough industry-level talk. Here’s what I’d tell any independent site owner, in four actionable points:

1. Ask three questions before buying any GEO tool

• Does this score have a denominator? (What’s it benchmarked against?)
• Can I see the raw data? (Or am I just handed a score?)
• Can this number translate into revenue? (If it goes up 10 points, what verifiable impact does that have on my orders?)

If a tool can’t answer all three, don’t buy it yet.

2. Treat prompt tracking like polling, not like a medical exam

Follow Kevin Indig’s framing: build your own prompt list covering your industry’s core scenarios (refresh it quarterly), run it regularly, and watch trends, not single snapshots. The value of polling is the long-term direction, not any one number.

3. Start free, then decide whether to pay

Google Search Console has already shipped a generative-AI performance report (Forrester and several practitioners note it’s not granular enough yet), and you can combine it with manual prompt runs and free AI citation checkers to build a rough measurement system at near-zero cost. Run it for three months. If you find trend-tracking genuinely needs automation, then look at paid tools.

4. Don’t treat tools as faith — treat trust as your moat

One more Kevin Indig finding worth pinning to your desk: 75% of consumers pick the number one result in an AI shortlist — but if they see a trusted brand anywhere on that list, they pick it. In the AI search era, brand trust is the hardest currency. Whether a tool can measure your visibility is a small matter; whether your brand deserves to be recommended by AI is the big one — and that’s built with long-term content, reputation, and first-party data assets, not by buying a fancier dashboard.

Wrapping up

This survey is essentially saying: the GEO measurement tool industry has a trust problem, not a pricing problem.

The data is valuable (4.20/5). The tools aren’t trusted (3.19/5). 57% say “I don’t believe the number”; only 7% complain about cost. Methodology opacity is a dead end baked into the business model, and the non-determinism and personalization of AI systems make “precise measurement” questionable in principle.

For independent site owners, the real takeaway is: don’t pay for anxiety. AI visibility measurement is a genuine need — but “measurement tools” are not the same as “measurement capability.” Figure out where the numbers come from, build your own low-cost measurement loop first, and invest in the brand trust that actually pays. Those things are worth more than any dashboard.

And for those vendors… I’ll leave you with Forrester’s closing line, which I’m passing along to every owner still hesitating on that checkout page:

The objection isn’t “it’s too expensive.” It’s “I don’t believe it” and “I can’t connect it to money.” And buying better data tomorrow won’t close either gap.