CueScoutBot: what it reads and how to block it

CueScoutBot is the page reader behind CueScout. Our customers track the questions their buyers ask ChatGPT, Perplexity, Gemini and Google's AI answers. When one of those engines cites a page in its answer, we fetch that page to see which brands it names.

If you found it in your logs, an AI engine cited your page for a question one of our customers follows.

One page per request: the address the engine cited. It does not follow links from that page or crawl the rest of your site.

It asks for HTML only, reads at most 1 MB, and gives up after 12 seconds. It does not run JavaScript, submit forms, log in, or send cookies.

For each customer, a page it has read is left alone for seven days. A page that failed to load is not tried again for three.

When a customer asks you to add them to a list, we read that one page again 3, 7, 14, 21 and 28 days later, to see whether it changed. That is five requests in four weeks.

If you answer that you will add them, or quote a price, we read it four more times over the four weeks after your reply, and four more if they pay for a placement. Then it stops.

For the same request we look once for a contact address you already publish: the author page the article links to, your homepage, and up to three pages the homepage links to, such as Contact or About. We only use an address printed on your own site. We never build one from a name and a domain.

It also reads our customers' own sites when it audits them, and a site someone enters into one of our free tools.

The page title and which of the brands our customer tracks appear in the text.

When a customer is weighing a paid placement on the page, a count of how it marks its outbound links: followed, nofollow or sponsored.

If the page names our customer, the one sentence that does, so they can see what we matched.

For a list a customer wants to be on, the author's name and a published contact address. Both are deleted 90 days after that request is closed.

We do not keep a copy of the page or republish any of it.

CueScout does not email publishers. If you got a note asking to be added to a list, a person at that company sent it from their own inbox. We drafted it for them from what the AI engines cited, and each account can only draft a limited number a month.

Return a 403 to any request whose User-Agent contains CueScoutBot, in your server or CDN rules. We record the page as unreadable and tell our customer exactly that. We do not report anything about what is on it.

It does not read robots.txt yet, so a Disallow line there will not stop it. The 403 will.

Questions, or something it did that this page does not describe: support@cuescout.com.