AI Crawler Census

AI Crawler Census

See if ChatGPT & Perplexity can read and cite your site

A free tool to check whether ChatGPT, Perplexity, and Google's AI answers can read and cite your site. Backed by an original crawler-blocking census of the top 5,000 sites' robots.txt rules (September 2026). No signup required.

Technology · Business & Finance

Features

Checks your site's robots.txt against real AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and more)

Backed by an original census of the top 5,000 sites' robots.txt rules

Shareable result page and an embeddable badge

Includes a free llms.txt validator

Use Cases

Site owners checking whether ChatGPT or Perplexity can cite their content

SEO/GEO practitioners auditing a client's AI-crawler exposure

Publishers deciding whether to allow or block AI training and search crawlers

Betula Assistant

Your second brain that gets things done. Call or text me.

Comments

reese-2

Built this after finding that 44% of sites blocking GPTBot also block OAI-SearchBot without realizing it's a different crawler with a different job. Wanted a free, no-signup way for anyone to check their own site in seconds, backed by real census data instead of a guess.

Upvote Reply

Premium Products

Face2social

Face search across public social media profiles

Wowmade.ai

AI Video, Image & Music Generator in One Creative Platform

TaroTown

AI Tarot Readers That Remember the Whole Story

Renoir AI

Create stunning images, videos, and audio with AI.

Continua AI

AI-powered investigations, grounded in evidence.

AirlineStream.dev

Airline logos for your app, delivered by API

Shora

Meditation for how you feel right now

Kipper

Ask your finance system questions in plain English

Reply200

Handle every comment: grow organic reach, sell under ads

CalculatorFox

Free online, embeddable calculators

ego lite

Fastest browser for AI agents

BillingHub

Pay per invoice, not per month. Credits never expire.

Backdrop

Live in-call sales assistant

ToolScope

Discover AI tools and create unique AI characters

PrompTessor

Generate, optimize, and manage AI prompts in one space

VoiceDrop

AI-Powered Ringless Voicemail & SMS

Ephedrine Powders

Ephedrine powder for sale, Ephedrine HCL, Buy Ephedrine powd

IZIPAY

IZIPAY is a crypto-funded virtual and physical card service

Zinn Hub® Freelance Marketplace

The Fairer Freelance Marketplace

Lockedweb - Porn Blocker

Porn blocker with no instant off switch.

FindTourGuide

Find the Right Tour Guide. Plan Better Trips.

StarterKitPro

StarterKitPro is your 90% done SaaS starter kit - all in Next.js

Mailvex

Build on-brand emails with AI and your brand kit

AI Tools List

Discover, compare, and explore the best AI tools

RELIABLE REFRIGERANT GAS FOR HVAC & REFR

Freon Refrigerant gas cylinders

NexPept

Lab-tested research peptides for Canadian laboratories

Friday

https://askfriday.io/

Velo Workspace

Disposable Linux/macOS VMs on Mac, with local AI Bridge

Buy

Betula Assistant

Your second brain that gets things done. Call or text me.

Zens AI

AI customer support that keeps improving your SaaS

EntreDash

Spark to business, your personal innovation OS

reese-2 · Follow

Makers

reese-2 · Follow