SEO & Webmaster
Processed using an external service
Paste robots.txt rules or fetch /robots.txt from a site, then test whether a URL path is allowed for a selected user-agent.
Robots parsing follows common crawler rules but may not match every search engine implementation exactly.
1. Paste robots.txt content or fetch it from your domain. 2. Choose a user-agent (Googlebot, Bingbot, or *). 3. Enter a path such as /blog/secret-page. 4. See whether the path is allowed or blocked before shipping a crawl change.
Example: Disallow: /private/ with User-agent: * blocks /private/report.pdf for all crawlers—test /public/ to confirm it stays allowed.
Paste robots.txt rules or fetch /robots.txt from a site, then test whether a URL path is allowed for a selected user-agent.
Robots parsing follows common crawler rules but may not match every search engine implementation exactly.
This tool sends data to an external service for processing. See the tool details below for what is transmitted.
Yes. Utilnivo tools are free to use and do not require an account.
Utilnivo does not require an account. See the privacy note on this page for how your data is processed for this tool.
Open the tool on this page, enter your inputs, and use the on-screen controls. No account is required.
Page last reviewed: