AI crawlers allowed in robots.txt
Site settings → SEO → robots.txt, then publish. Webflow only serves your robots.txt on a custom domain — on a .webflow.io subdomain it serves its own, which blocks everything.
Site platforms
Webflow, and what an answer engine can read of it
Webflow sites start ahead of most of the category on the checks that matter: pages are static HTML, so everything on them is readable without JavaScript, and the SEO panel sets titles, descriptions and canonicals properly. What Webflow does not give you is the site root. There is no directory to drop a file into, which decides two of the eight checks before you start.
The failure worth knowing about first is the robots.txt one, because it is silent. Webflow only serves the robots.txt you wrote once the site is on a custom domain — on a webflow.io subdomain it serves its own, which disallows everything. A staging site that an engine has been crawling for months has been crawling nothing.
How we recognise it: a Webflow generator tag, the data-wf-page attribute, or assets on Webflow's CDN. Run the free readiness check and the fixes it gives you are the Webflow ones below.
What to do
1.Check what your live robots.txt actually says
Not what the settings panel says — what the domain serves. If this comes back with a blanket Disallow and you did not write one, you are looking at Webflow's subdomain default, and the fix is the custom domain rather than the setting.
In a terminal
curl -s https://yourdomain.com/robots.txt2.Put the Organization block in site-wide head code
Site settings → Custom code → Head code. The sameAs list is what ties your brand name to an entity the engines already have a record of; without it you are a string that happens to appear on a page. Use the profiles you actually control.
Site settings → Custom code → Head code
<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "Organization", "name": "Your company", "url": "https://yourdomain.com", "sameAs": [ "https://www.linkedin.com/company/yourcompany", "https://www.crunchbase.com/organization/yourcompany" ] } </script>3.Give the FAQ markup a page that already answers questions
Page settings → Custom code on that page, not the site-wide slot — FAQPage markup describing questions that are not on the page it loads on is the kind of thing that gets a site's markup ignored wholesale. Webflow's accordion component emits no schema of its own, so this is written by hand either way.
4.Decide whether /llms.txt is worth a proxy
It cannot be a Webflow page: every path returns HTML, so a page slugged llms.txt serves a web page rather than a text file, and the check reads it as absent. A Cloudflare Worker in front of the domain answering that one path is about ten lines. If you are not already proxying, this is a warn worth leaving alone — no engine refuses to read a site without one.
The readiness checks on Webflow
CueScout runs eight checks on whether an answer engine can read a site at all. These are the ones whose answer is different on Webflow — the rest are the same everywhere.
llms.txt present and parseable
Not possible hereWebflow serves HTML for every path, so a page whose slug is llms.txt still comes back as a web page rather than plain text. The working route is a reverse proxy on your domain (a Cloudflare Worker on /llms.txt is the usual one).
Organization schema with sameAs links
Site settings → Custom code → Head code, paste the Organization JSON-LD in a <script type="application/ld+json"> block, then publish. Per-page code lives in that page's settings instead.
FAQPage schema present
Page settings → Custom code → Head code on the page that answers the questions. Webflow will not generate this from your accordion component — the markup has to be written.
Canonical, title and meta description set
Page settings → SEO, per page. Webflow sets the canonical for you unless you have overridden it in that same panel.
These describe Webflow’s own settings, which move without warning us. Last checked against the product on 2026-08-14.
What this does not do
- You cannot publish a plain-text file at the site root. /llms.txt needs a reverse proxy in front of Webflow or it does not exist.
- Custom code is a paid-site feature. On a free Webflow site the schema checks cannot be passed at all.
- Webflow's own robots.txt on a .webflow.io subdomain overrides anything you write, and it disallows everything.
- CueScout reads your site from the outside. It does not connect to your Webflow account, cannot publish for you, and never sees your project.
Frequently asked questions
Does CueScout connect to my Webflow account?
No. Every check is an HTTP request to your public site, the same as any crawler makes. There is no OAuth, no app to install, and nothing that could change your project.
Why does the crawler check fail when my robots.txt looks right in the editor?
Almost always because the site being checked is a webflow.io subdomain. Webflow serves its own robots.txt there — a blanket disallow — and only serves yours once a custom domain is attached and published.
Is a Webflow site at a disadvantage for AI citations?
Not meaningfully. The two checks it makes hard are llms.txt and, on free sites, schema. Neither is what decides whether an engine recommends you — that is mostly about which pages elsewhere on the web mention your category, which is the same problem on every platform.
Related
Site platforms
Every integration we publish Framer
The robots.txt setting, custom head code on paid sites, and why /llms.txt needs a proxy.
Site platformsSquarespace
Two checks Squarespace makes impossible, and the schema ones you can still pass through code injection.
Site platformsWordPress
The virtual robots.txt, the Yoast and Rank Math settings that produce sameAs, and where llms.txt goes.
See where your Webflow site stands
The readiness check is free, needs no signup, and takes about half a minute. It works out what your site is built on and gives you the fixes written for it.