Generate a clean robots.txt block that lets AI engines in without breaking your existing rules. Toggle bots, set sitemap, block admin paths.
All selected bots will be explicitly allowed.
Applies to User-agent: *.
Optional. Use only if you want to block a specific bot.
# robots.txt — generated by BrandCitation # https://brandcitation.com/tools/ai-robots-txt-generator # # AI search engines and assistants User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Googlebot-Extended Allow: / User-agent: Applebot-Extended Allow: / User-agent: Amazonbot Allow: / User-agent: meta-externalagent Allow: / User-agent: Bytespider Allow: / User-agent: Cohere-AI Allow: / User-agent: DuckAssistBot Allow: / # Path blocks (recommended) User-agent: * Disallow: /admin Disallow: /wp-admin Disallow: /login Disallow: /private Disallow: /cart Disallow: /checkout Disallow: /account Allow: / # Sitemaps Sitemap: https://example.com/sitemap.xml
How to deploy
/robots.txt at the root of your domain.https://yourdomain.com/robots.txt.API-based. GDPR-aware. We never train models on your data.
The AI Crawler Audit checks robots.txt + 30 other signals and ships a 30-day plan.
Request the $149 AI Crawler Audit