AI crawlers allowed in robots.txt
Mintlify's default robots.txt allows the AI crawlers. A block here comes from whatever proxies your docs domain, not from Mintlify.
Site platforms
Mintlify is the only platform on this list that ships the AI-readiness work by default. It generates /llms.txt and /llms-full.txt from your docs, its robots.txt lets the AI crawlers in, and pages are pre-rendered. A Mintlify docs site passes checks that most sites need a project to reach.
What is left is what documentation platforms leave everywhere: no Organization schema, and no route from the docs to the commercial pages. Assistants that land on your docs — which they do constantly, because docs answer questions — learn what your product does and nothing about what it costs or who makes it.
How we recognise it: assets on Mintlify's CDN, or a Mintlify attribution link. Run the free readiness check and the fixes it gives you are the Mintlify ones below.
If the docs live at docs.yourdomain.com, that is a separate site as far as every check is concerned — its own robots.txt, its own llms.txt, its own schema. Run the check against both hosts, because they will not have the same answers.
The head field takes raw tags. This is the cheapest win on a docs site: it turns the name in your header from a string into a company the engines can connect to everything else they know about you.
docs.json
{
"head": [
"<script type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"Organization\",\"name\":\"Your company\",\"url\":\"https://yourdomain.com\",\"sameAs\":[\"https://www.linkedin.com/company/yourcompany\"]}</script>"
]
}An assistant that arrives at your docs from a how-do-I question can only follow links that exist. If the docs are an island, everything it learns about you is technical, and the buying question gets answered from somebody else's comparison page.
CueScout runs eight checks on whether an answer engine can read a site at all. These are the ones whose answer is different on Mintlify — the rest are the same everywhere.
Mintlify's default robots.txt allows the AI crawlers. A block here comes from whatever proxies your docs domain, not from Mintlify.
Mintlify generates /llms.txt and /llms-full.txt from your docs automatically. If this is failing, the docs site is not being served at the root of the domain we checked.
docs.json → the head field (mint.json → analytics/head on older setups) takes raw tags, including a JSON-LD block. Docs sites rarely carry Organization schema and it is one of the cheaper wins.
Your docs domain has no pricing page and does not need one — but assistants that land on your docs never find your price. Link it from the docs nav so the crawl reaches it.
These describe Mintlify’s own settings, which move without warning us. Last checked against the product on 2026-08-14.
No, for the docs domain. Your marketing site is a separate host and Mintlify does nothing for it — that one is on you.
Heavily, for how-to and integration questions, because docs answer them precisely and are usually well-structured. Far less for buying questions, which get answered from roundups and community threads instead.
For this narrow question, a subdirectory keeps everything on one host and one set of signals. It is also a migration, and the citation difference is not large enough to justify one on its own.
Docusaurus
static/ handles the files, headTags handles the schema, and pre-rendering handles the rest.
Site platformsNext.js
app/robots.ts, a route handler for llms.txt, JSON-LD in the layout, and the client-component trap.
Site platformsGhost
robots.txt lives in the theme, llms.txt goes in /public, and Ghost already writes half the schema.
The readiness check is free, needs no signup, and takes about half a minute. It works out what your site is built on and gives you the fixes written for it.