Being Mentioned in AI Answers Is Not the Same as Being Recommended

Every AI visibility tool counts mentions. Ours does too, and I've started to think the number has been flattering us.

Here's what made me look. I'd been reading that dashboard for months without ever clicking through to an actual answer, which in hindsight is a strange thing to admit. When I finally did, this is roughly what was in there: a buying question goes to ChatGPT, and the answer names four products, compares them for a paragraph, then declines to pick one. "Mixpanel, Amplitude and Heap are what most small teams end up on. Which one wins depends on whether you need session replay in the same tool."

Three brands named. Nobody recommended. My dashboard called that three wins, and for the person who asked it's a non-answer, because they still have to go and choose.

Four things being named can mean

Once you read the answers instead of the counts, one fact splits into four:

• Recommended. The answer steers the reader to you, either outright or for a situation it describes.
• Listed only. You're named, and it advises someone else. Or nobody.
• Dismissed. You're named, and it argues against you.
• Absent. You're not in there at all.

A mention count can only separate the last one from the other three. The first three score the same and they're worth wildly different amounts. Being dismissed by name is worse than being absent, I'd say, because the reader now has a reason not to consider you and your dashboard has filed it under presence.

This isn't an edge case either. Roundups, category pages, "best tools" listicles: those are the page shapes engines reach for hardest on buying questions, and their whole job is to name everybody. If that's where your visibility comes from, you're being read, summarised and skipped.

Why not just prompt a chat model

That was my first attempt, for the record. Hand every stored answer to a chat model, ask who it recommended. It works fine until you try to count the results.

The replies come back as prose, and slightly different prose every time. "It leans toward Mixpanel", "Mixpanel is the clear recommendation here", "the answer recommends Mixpanel for larger teams, though it hedges" — three sentences that all have to collapse into one countable fact, and suddenly half your engineering is that parsing step. Then someone tweaks a prompt and the parser starts mis-reading a case it used to handle fine, with nothing failing loudly enough to notice.

So I used a decision model instead. Jev, through OpenRouter. It returns probabilities rather than text: you hand it a state and a set of questions, each with the answers you'll accept, and it gives you a distribution across those options. It can't write a sentence. It can't invent an option you didn't list. For counting things, that limitation turns out to be the whole point.

Four questions per stored answer:

1. Which single brand does this answer steer the reader towards? Options are the brand in question, each tracked competitor, none, and other.
2. Does it advise a choice at all, or just list names and leave it open?
3. How does the brand show up: recommended, listed, dismissed, absent.
4. What kind of source is it leaning on: listicle, community thread, comparison page, review site, docs, vendor page.

Four typed answers. No prose, nothing to parse.

What it cost

The corpus was our demo workspace — a fictional product-analytics brand, twelve buying questions across ChatGPT, Perplexity and Gemini, 63 distinct stored answers between them.

63

answers judged

252

judgments

7.28s

wall clock

$0.0033

OpenRouter spend

typesafe/jev-1.13 via OpenRouter, 78,850 in / 13,744 out tokens. Works out to $0.0000131 per judgment, about $13 per million, and $0.0000524 per answer for all four questions.

I re-read that twice because I assumed I'd misplaced a decimal.

The cost is what shifted my thinking, more than the speed. Re-reading every stored answer a second way used to be one of those things I'd scope, cost out, and quietly drop. At three tenths of a cent it stops being a project and becomes a flag you turn on.

What the split looked like

Two readings over the same 63 answers. The left one is our existing matching rule, untouched. The right one is the judgment.

Mixpanel

named100.0%

recommended3.2%

BrightMetric

named63.5%

recommended44.4%

Amplitude

named50.8%

recommendednever

Heap

named33.3%

recommendednever

PostHog

named15.9%

recommendednever

52.4% of answers recommended nobody at all.

Share of 63 stored answers. Demo workspace, not a market reading. ChatGPT, Perplexity and Gemini, 21 September 2026. Named is a string match and is unchanged; recommended is the classifier reading the same text.

The biggest row isn't a brand at all. 52.4% of answers recommended nobody: named some products, handed the decision back.

Now, before anyone quotes those percentages at me. This is our demo corpus, which means it describes our fixtures and not the product-analytics market, and the answers in it were written by us. The method is what I'm publishing. The engineering figures above are real measurements of a real run; the market figures are not figures about a market.

What I'd expect to survive contact with real data is the shape rather than the values — that the most-mentioned brand in a category is rarely the most-recommended one, and that a lot of buying answers recommend nobody. I don't know that yet. If you run this on your own corpus I'd genuinely like to hear whether it holds.

The part we deliberately didn't do

We didn't let the model decide who was named.

Named is still a matching rule: your brand string is in the text, or it isn't. Cited is still a URL match against what the engine handed back as sources. No model adjudicates either one, and none of this work touched that code.

Which cost us some elegance, honestly. It would have been tidier to ask for everything in one pass, and I did consider it. But when someone opens their dashboard and their mention rate has moved, "the string appeared in the text" is an explanation you can walk through line by line, and "the judgment changed this week" isn't. Trust is something you spend rather than something you have.

So the judgment sits beside the rules and never on top of them. Two readings, two methods, and the gap between them is the part worth looking at.

If you're named but never picked

The useful question isn't how to get mentioned more. You're already mentioned. It's which pages are producing the answers that skip you.

So collect the cited URLs for the questions where you were named and not advised. In our own audits the pattern is consistent enough to plan around: a handful of hosts supply most of a category's answers, and they're rarely your blog. Three shapes keep coming up.

The roundup you're missing from. The engine read a "best X for Y" article, it lists eight products, you're not one of them. Writing a better post on your own domain doesn't fix that. Getting a row added does. Worth knowing before you draft the pitch: those placements are frequently paid, and treating them as free link building burns a week — I wrote up the approach in how to pitch a roundup post inclusion.

The comparison page that describes you wrong. You're on it, and the description comes off a pricing page you replaced eight months ago. The engine repeats it faithfully. Cheap to fix, and almost nobody checks.

The community thread. Perplexity leans on these hard. Across one 36-question run it cited Reddit 38 times out of 346 citations, while ChatGPT cited it zero times out of 139 (full breakdown here). The honest play is being the page that thread links to, not going and posting in it.

If you haven't collected cited URLs at all yet, start there instead of here. How to audit brand mentions in AI engines covers that pass, and none of this second reading makes sense until you've got answers stored to read.

Caveats

One run, one corpus, one shape of problem. Not a benchmark, and I wouldn't extrapolate the timing to yours.

The threshold is a guess and I want to flag it properly. Somewhere between "this clearly advises a choice" and "this merely leans that way" there's a line, and we put it in code rather than in the stored row so it can move once there's enough real data to argue about. If you build something like this, expect to move yours a few times.

And the measurement isn't the work. Knowing you're named in most answers and advised in few tells you where you stand, which is useful for about a day. Changing it means changing the pages the engines read, which is slower, much less satisfying, and the only thing that moves the second column.

The board this run produced is at /tools/category-teardown, cost printed from the run rather than typed in.