Robots.txt & Sitemap Validator

The robots.txt file sits at the root of every website and acts as a set of instructions for search engine crawlers, telling them which pages they are allowed or not allowed to visit. A common and costly mistake — especially when launching a new site after development — is forgetting to remove a Disallow: / rule that blocks all search engines from indexing the site entirely.

Enter any domain below to fetch and inspect its robots.txt file, check for dangerous rules, and validate its sitemap. If you recently moved from a staging environment to production, check this immediately.