Free tool · No sign-up
Robots.txt & Sitemap Checker
Check if a public robots.txt responds and identify sitemap URLs advertised there.
Check the first crawler file at your domain.
Enter your homepage address and we will check the public robots.txt response and sitemap declarations.
How to read the result
Robots.txt and sitemaps do different jobs
A sitemap is a list of URLs you want search engines to discover. Robots.txt is a set of crawler instructions published at the site root. They are related, but they are not interchangeable.
This checker fetches the public robots.txt at the domain you enter and looks for the HTTP response plus sitemap declarations. It helps you answer a narrow question: is the file there, and is it pointing crawlers toward the sitemap locations you intended?
Do not use robots.txt as a hiding place
Robots.txt is not a secure access-control mechanism. If something must not be public, protect it properly. Also remember that a sitemap does not force indexing. A URL still needs useful, accessible content and must satisfy search engine requirements.
When to investigate further
If the file returns an unexpected status, contains a rule you do not recognise, or points to an old sitemap, fix the configuration and then confirm the result in your search tools.