robots.txt monitoring fetches your robots.txt file on each check cycle and alerts you when the content changes. An accidental Disallow: / can block all search engines within hours.
⚙️How it works
Uptrue fetches /robots.txt from your domain and computes a SHA-256 hash of the content. If the hash differs from the previous run, a change alert fires.
✓What Uptrue checks
- robots.txt accessibility (HTTP status)
- Content hash change detection
- File size
⚠Alert conditions
- robots.txt content changes from baseline
- robots.txt returns non-200 status
Why this matters
A single line change in robots.txt can tell Google to stop crawling your entire site. This has happened to major brands during CMS migrations and framework upgrades.
Set up in 60 seconds
Free plan · 3 monitors · No credit card required
?Frequently asked questions
Does it detect what changed?
It alerts you that content changed and gives you the hash. You'll need to compare the current file to your baseline.
What if my robots.txt is intentionally dynamic?
You can pause the monitor or increase the check interval if frequent changes are expected.
Related Monitor Types
🤖
Ready to set up robots.txt Change Monitoring?
Join teams who monitor their infrastructure with Uptrue. Free plan, no credit card required. Want to spot-check first? Run our free robots.txt checker.
Start Monitoring Free