Free tool

Can ChatGPT, Claude and Perplexity read your website?

This free AI visibility checker fetches your page the way AI crawlers do, without JavaScript, then checks robots.txt for 20 AI crawlers, indexing rules, the text and structure machines read, and the signals that tell AI who you are. You get a score out of 100 and the five fixes that matter most, in about ten seconds.

Free · no sign-up · checks one page plus your robots.txt and sitemap

What we check

Three questions decide whether AI can cite you.

More than 20 checks, weighted by how much each one changes whether AI search can use your page.

  • Can AI crawlers reach the page?

    If a crawler may not fetch a page, or the page says “do not index” or “do not quote”, nothing else matters.

    • robots.txt rules for 20 AI crawlers, from OAI-SearchBot to PerplexityBot
    • noindex, nosnippet, max-snippet, and Bing’s noarchive and nocache, in meta tags and X-Robots-Tag headers
    • HTTPS, redirects and server response time
    • An XML sitemap, in robots.txt or at /sitemap.xml
  • Can AI read and understand it?

    We read the HTML your server sends without running JavaScript, the way most AI crawlers do.

    • How much text is in the HTML, and whether the page is an empty JavaScript app shell
    • Title, meta description and one clear H1
    • Sections, question-style headings and an answer-first opening paragraph
    • Lists and tables, the page language and image alt text
  • Does AI know who you are?

    AI assistants recommend businesses they can identify and trust, so entity signals count.

    • Valid JSON-LD structured data and its types
    • Organization or LocalBusiness data, with sameAs links to your profiles
    • About and contact pages, and dates that show the page is kept up to date
    • Canonical URL and Open Graph link previews

Reference

AI crawlers and what they do.

The robots.txt names of the AI crawlers this tool checks, from each company's own documentation (November 2026). Search and user crawlers decide whether you can be cited; training crawlers do not.

AI crawler user agents, vendors and purposes
User agentCompanyUsed forWhat it does
OAI-SearchBotOpenAIAI search and citationsSurfaces sites in ChatGPT search. OpenAI: sites that block it are not shown in ChatGPT search answers. Docs
ChatGPT-UserOpenAIFetches a page a user asks aboutOpens a page when a ChatGPT user asks. OpenAI says robots.txt may not apply, and it does not decide search inclusion. Docs
GPTBotOpenAIModel trainingBlocking it says your content should not be used to train OpenAI’s models. It does not affect ChatGPT search. Docs
Claude-SearchBotAnthropicAI search and citationsIndexes pages for search in Claude. Blocking it may reduce your visibility in Claude’s answers. Docs
Claude-UserAnthropicFetches a page a user asks aboutFetches a page when a Claude user asks about it. Anthropic says its bots honour robots.txt. Docs
ClaudeBotAnthropicModel trainingBlocking it excludes your future pages from Anthropic’s training data. Docs
PerplexityBotPerplexityAI search and citationsSurfaces and links sites in Perplexity answers; Perplexity says it is not used to train AI models. Docs
Perplexity-UserPerplexityFetches a page a user asks aboutFetches a page a user asks about. Perplexity says it generally ignores robots.txt. Docs
GooglebotGoogleAI search and citationsGoogle’s search crawler. Google: robots.txt rules for Googlebot are the control for Search, including its AI features. Docs
Google-ExtendedGoogleModel trainingA control token, not a crawler: whether Gemini may train on or ground answers with your pages. It does not affect Google Search. Docs
BingbotMicrosoftAI search and citationsBing’s crawler. Microsoft says Copilot sends search queries to Bing for web answers. Docs
ApplebotAppleAI search and citationsPowers search in Siri, Spotlight and Safari, and may render JavaScript. Apple may also use what it crawls for training. Docs
Applebot-ExtendedAppleModel trainingDoes not crawl. Blocking it opts pages Applebot crawled out of Apple’s model training, without affecting search. Docs
DuckAssistBotDuckDuckGoAI search and citationsFetches pages in real time for DuckDuckGo’s AI-assisted answers; not used to train models. Docs
Amzn-SearchBotAmazonAI search and citationsSearch crawler for Amazon experiences such as Alexa; Amazon says it is not used for training. Docs
AmazonbotAmazonModel trainingAmazon’s general crawler; Amazon says it may be used to train its AI models. Docs
meta-webindexerMetaAI search and citationsCrawls to improve Meta AI’s search results. Docs
meta-externalagentMetaModel trainingCrawls for uses such as training Meta’s AI models. Docs
MistralAI-UserMistralFetches a page a user asks aboutFetches pages when a Mistral user asks; robots.txt governs which sites it may fetch. Docs
CCBotCommon CrawlModel trainingBuilds the free Common Crawl web archive, which many AI training datasets draw on. Docs

Of these, 13 fetch pages for AI search answers or when a user asks about a page; blocking the search crawlers can keep you out of those answers. Several vendors say their user-triggered fetchers may not follow robots.txt at all. Blocking a training crawler only keeps your pages out of future model training.

Copy and adapt

A robots.txt that allows AI search and opts out of training.

# Let AI search engines and assistants read and cite the site
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: DuckAssistBot
Allow: /

# Optional: opt out of model training
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: meta-externalagent
User-agent: Amazonbot
User-agent: CCBot
Disallow: /

# Everyone else, including Googlebot and Bingbot
User-agent: *
Allow: /

Sitemap: https://www.example.com/sitemap.xml

Rules for a named crawler replace the * rules for that crawler, so list any private paths in every group. Leave out the training block if you are happy for your public pages to be used for training. Google-Extended and Applebot-Extended are control tokens, not separate crawlers: they never fetch pages, they only tell Google and Apple how pages that Googlebot or Applebot crawled may be used.

Be clear-eyed

What a score cannot tell you.

  • What AI answers actually say

    The checker does not ask ChatGPT or Perplexity about you. Track that with a fixed list of the prompts your buyers use, every month.

  • What other sites say about you

    AI assistants lean on third-party sources: directories, review sites, press and forums. Profiles and reviews there matter as much as your own pages.

  • Firewall and CDN blocking

    A CDN or firewall can block AI crawlers before robots.txt matters. Cloudflare, for example, has AI crawler settings and can manage robots.txt for you, adding blocks for training crawlers. Check those settings if a crawler should be allowed.

FAQ

AI visibility questions.

What is AI visibility?

AI visibility is how likely AI assistants such as ChatGPT, Claude, Perplexity, Gemini, Copilot and Google’s AI Overviews are to find, read and cite your website when people ask about what you do. It has two halves: technical access (can their crawlers fetch and understand your pages?), which this checker tests, and reputation (what other sites, directories and reviews say about you), which no page-level tool can measure.

How does the AI visibility checker work?

It fetches the page you enter the way an AI crawler does, without running JavaScript, and reads your robots.txt and sitemap. It runs more than 20 checks in three groups (access, content, identity), weights them by impact, and gives a score out of 100 with the five fixes that matter most. A blocked AI search crawler or a noindex tag caps the score at 39, because nothing else helps until that is fixed. No AI model is involved: every check is a rule you can verify yourself.

Should I block GPTBot and other AI training crawlers?

That is a business choice, and it is separate from AI search. OpenAI and Anthropic document separate crawlers for search and for training: OpenAI says sites that block OAI-SearchBot are not shown in ChatGPT search answers, while blocking GPTBot only keeps your content out of model training. Google-Extended works the same way and, per Google, does not affect Google Search. So you can block training crawlers and still allow the search crawlers that decide whether you can be cited. The sample robots.txt on this page does exactly that.

Do AI crawlers run JavaScript?

Most do not. Vercel’s study of AI crawler traffic (December 2024) found that none of the major AI crawlers it measured, including OpenAI’s, ClaudeBot and PerplexityBot, render JavaScript; Googlebot and Applebot do. Text that only appears after JavaScript runs is invisible to the rest. If the checker finds little text in your HTML, render pages on the server or pre-render them.

Does llms.txt help AI visibility?

Not as far as anyone can show. llms.txt is a proposed file that summarises a site for language models. It is harmless and quick to add, but none of the major AI providers says its crawlers read it, and Google’s guidance on AI features says you do not need new machine-readable or AI text files to appear in them. The checker reports whether you have one without counting it in the score.

If my score is high, will ChatGPT recommend my business?

Not by itself. A high score means AI crawlers can reach, read and identify your pages, which is the part you control on your own site. Whether an assistant recommends you also depends on what other sites say about you: directory profiles, reviews, press and forum mentions. Our SEO & GEO Agent tracks both, including which prompts cite you and which cite competitors.

Is the checker free, and what do you store?

It is free, with no sign-up. We fetch the page, robots.txt, sitemap and llms.txt once, keep the result in memory for 10 minutes so a repeat check is instant, and log only the host name and the score. A Cloudflare Turnstile check keeps bots from abusing the tool. Nothing reaches our team unless you press “Send this report”. See the privacy policy.

Want AI search to recommend you, not just read you?

We fix the technical side, build the third-party profile, and track your citations every month.

NDA on request from the first call · You own 100% of the code and IP

Visionbotix

AI concierge · answers from this website

Live · sandbox
  1. Hi! I'm Visionbotix's AI concierge. Ask me what we build, what it costs, or which agent fits your business. I answer from this website and can set up a call with the team.

Live AI · fictional business · don't share real personal data · how demo chats are handled