Check if AI can read your site
Paste your address and this tool reads your robots.txt, works out what it allows and blocks for thirteen AI agents, and compares how your site answers a browser against how it answers GPTBot’s user agent. No email, no account, nothing stored.
Published 8 September 2026
Short answer
A site can rank first on Google and be unreadable to ChatGPT. The cause is rarely robots.txt: it is a bot-protection rule at the CDN returning a challenge to user agents it does not recognise, while letting Googlebot through because it is allow-listed by default. This tool checks both layers and tells you which one is failing.
DoGeo is an SEO and GEO agency. This tool is free, needs no account, and is not the service: the service is the work, and we do that ourselves.
What it checks, and what it does not
ScopeDoGeo would rather be explicit about the limits of a free tool, because a test that promises more than it delivers buys a false sense of safety, and that gets expensive.
What it covers
both layersIt reads your actual robots.txt and applies the grouping rules the crawlers themselves use: the agent’s own group where one exists, the wildcard otherwise, and within the group the longest matching pattern wins. Then it requests your page twice at the same moment, once as a browser and once as GPTBot, and compares the status codes.
What it misses
the honest limitsThe CDN probe uses one agent and one URL. A block can affect only a different agent, only other paths, or trigger on request frequency rather than user agent. It also cannot see whether your content depends on JavaScript, which is the second most common failure.
The definitive test
your logsThe evidence that settles it is hits by user agent in your own server logs over thirty days. Zero hits from an agent you claim to allow means the block is at another layer, or that nothing ever found you. That is part of the audit, not of this tool.
How to read the result
InterpretationThe three verdicts mean different things, and one of them is constantly misread.
Explicitly allowed
✓That agent has its own block in your robots.txt and no rule blocks the path. It is the clearest state: somebody made the decision deliberately and it is written down.
Allowed by default
✓No rule applies to it, so it may crawl. This is correct and it is the normal case. Its only weakness is that it records no decision, which is why DoGeo names every agent in its own robots.txt even though it does not have to.
Blocked
✕A rule prevents it, and the tool tells you which rule and which group it sits in. When nobody on the team knows who added it, it almost always came from a copied template or from two-year-old advice about protecting content from training.
The result that matters most is not in the agent list: it is in the CDN comparison. If your site answers 200 to a browser and 403 to GPTBot, your robots.txt can be immaculate and no assistant will be able to cite you anyway.
What to do when something comes back blocked
Next stepDoGeo recommends this order, which is the order the audit follows.
- Find out who added the rule and why before removing it. Sometimes there is a real reason, usually paid content, and the right fix is then to block a path rather than the whole site.
- If the block is in robots.txt and there is no reason for it, remove it and replace it with an explicit Allow naming the agent, so the next person to read the file knows it was a decision.
- If the block is at the CDN, the rule lives in your bot-protection product (Cloudflare, AWS WAF, Fastly or equivalent) and those user agents have to be added to its allow list. This is where most people fail, and it cannot be fixed from robots.txt.
- Then check whether your content exists before JavaScript runs: disable JavaScript in a browser and reload. Unblocking a crawler that will only find an empty page changes nothing.
- Come back to your server logs in thirty days. Hits by user agent are the only confirmation that the fix worked.
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Google-Extended
Allow: /Where to go next
And if it comes back clean and you still never appear?
Then the problem is further down: in rendering, in how your pages are written, or in what outside sources say about you. DoGeo checks it by hand and reports what it finds. Four things and we can tell you where you stand.