Free LLM Presence Checker
Check whether your website is accessible to ChatGPT, Claude, Perplexity, and Google Gemini by analyzing your robots.txt.
We'll attempt to fetch your robots.txt and llms.txt. If CORS blocks the request, try manual mode.
How to use this tool
Enter your website URL or paste your robots.txt content
Click "Check Presence" to scan for AI-specific user-agent rules
Review which AI platforms can access your content and follow recommendations
Why AI crawler access matters
AI-powered search is on the rise. ChatGPT, Claude, Perplexity, and Google Gemini are increasingly used to answer questions and surface content. When users ask these platforms a question, they rely on indexed web content — and if your site blocks their crawlers, you're invisible.
Your robots.txt file controls which crawlers can access your site. Each major AI platform uses a specific user-agent (e.g., GPTBot, ClaudeBot, PerplexityBot, Google-Extended). A simple Disallow: / under one of these agents blocks that platform entirely.
Blocking AI crawlers may seem like a way to protect your content, but it often hurts visibility. For content sites and businesses, allowing AI platforms to index your work can drive discovery and traffic. This tool helps you see exactly where you stand.
Frequently asked questions
How does this checker work?
It parses your robots.txt for AI-specific user-agent rules, including GPTBot, ChatGPT-User, ClaudeBot, anthropic-ai, PerplexityBot, and Google-Extended. It checks whether each platform is allowed, partially restricted, or fully blocked.
Should I allow all AI crawlers?
Generally yes for content sites. Allowing AI crawlers helps your content appear in AI-powered search and answers. Blocking them reduces your visibility when users ask ChatGPT, Claude, or Perplexity questions that your content could answer.
What's the difference between robots.txt and llms.txt?
robots.txt controls crawling permissions — it tells crawlers what they may or may not access. llms.txt provides structured content for AI consumption, listing your best content and metadata. Both matter for AI discoverability.
How do I fix blocked crawlers?
Remove or modify the Disallow rules for AI user-agents in your robots.txt. For example, delete the block for GPTBot if you want ChatGPT to index your site, or add Allow rules for specific paths you want to make available.
More free tools
llms.txt Generator
Create a standards-compliant llms.txt file for AI discoverability.
OpenGraph Preview
Preview how your pages look when shared on social platforms.
Keyword Density Checker
Analyze keyword frequency to optimize for search engines.
JSON-LD Schema Generator
Generate BlogPosting structured data for search engines.
Want this automated?
Attract handles AI discoverability, llms.txt, robots.txt best practices, and SEO — automatically for every blog it powers.
Pay as you go with credits. Autopilot $49/mo optional.