Robots.txt Generator
Create a properly formatted robots.txt file to control crawler access to your website. Allow or disallow specific bots, set crawl-delay, and point to your XML sitemap.
Configure Your robots.txt
Generated robots.txt
User-agent: *
Disallow:
About robots.txt
robots.txt tells search engine crawlers which pages they can or cannot access. Place the generated file in your website's root directory.