A sample is loaded. Open yoursite.com/robots.txt in a new tab, copy everything, and paste it here. Results update as you type.
| Crawler | Purpose | Homepage access |
|---|
Get a ready-to-use robots.txt block
Add this at the end of your robots.txt. Blocking training bots does not stop AI search bots from citing you.
Also try: Free llms.txt Generator
Want AI to recommend your brand?Digital Findr helps businesses show up in ChatGPT, Perplexity and Google AI answers.
Talk to Digital Findr