FREE · NO SIGNUP · NO STORED REPORT

Can AI search
crawl your site?

Test the robots.txt rules that apply to ChatGPT search, Claude search, Perplexity search, and Google. This checks permission to fetch your homepage—not whether an answer engine has indexed, ranked, or cited it.

We fetch the public homepage and its robots.txt. Submitted URLs and results are not stored.

THE SHORT ANSWER

Robots access is a gate,
not a visibility score.

If a relevant search crawler is disallowed from `/`, the crawler cannot retrieve the homepage under those rules. If it is allowed, that removes one technical barrier but does not establish indexing, recommendation, or citation.

OFFICIAL AGENTS

What the four checks mean

OAI-SearchBot

OpenAI says publishers should allow this agent for content to be discoverable and included in ChatGPT search summaries and snippets.

OpenAI documentation →Claude-SearchBot

Anthropic identifies this as the crawler that indexes content to improve relevance and accuracy in Claude search results.

Anthropic documentation →PerplexityBot

Perplexity identifies this as the crawler used to surface and link websites in its search results.

Perplexity documentation →Googlebot

Googlebot crawls content for Google Search, whose indexed pages can also support Google's AI search experiences.

Google documentation →

READ THE RESULT CORRECTLY

Allowed does not mean cited.

  1. 01
    Robots rules

    This tool evaluates the homepage path against the returned user-agent groups, Allow rules, and Disallow rules.

  2. 02
    Network access

    A CDN or firewall can still deny a legitimate crawler even when robots.txt allows it. Server logs are the stronger evidence.

  3. 03
    Retrieval and selection

    Indexing, relevance, authority, answer quality, and engine-specific systems determine whether a page is retrieved or cited.

ACCESS IS ONLY STEP ONE

Make the page useful
after it can be fetched.

AnswerReady rewrites and structures five existing revenue pages for clear extraction, evidence, and buyer action. Scope is confirmed before payment.

Request a $149 scope review →