AI's Wide Shut code · Part 3 — Stop Feeding the Machine
Chapter 9 Your Public Content and AI Crawlers
If You Have Your Own Website
# p. 45
At the top level of your site, so it's at yourdomain.com/robots.txt.
# Block AI training crawlers
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: Applebot-Extended
Disallow: /
User-agent: Meta-ExternalAgent
Disallow: /
User-agent: CCBot
Disallow: /The up-to-date version of this file, kept current on this site →
Reproduced exactly from AI's Wide Shut, Chapter 9. The book explains what each line does and what to do when it doesn't work. About the book
Something here not working as the book describes? Tell the author.