robots.txt generator

Configure allow/deny rules per crawler group and generate robots.txt. All processing runs in your browser; nothing is uploaded to our servers.

Site type

Disallow / Allow rules for wp-admin are included automatically.

Sitemap URL

Additional Disallow paths

Applied to User-agent: * (one path per line)

Presets

0/6 categories blocked

Search enginesTraditional search engine crawlers

AI trainingLLM training data collection

AI searchAI search & live fetch on user action

OtherMixed-purpose crawlers

SEO toolsSEO audit & analysis crawlers

Social & previewsSocial link previews & OGP fetch

Output

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Note
robots.txt is only a request to crawlers, not legally binding. Malicious bots may ignore it. For stronger blocking, combine with Cloudflare WAF or similar.