SEO Utilities

Robots.txt Tester

Test robots.txt allow and disallow rules against URL paths and crawler user agents.

robots.txt testerrobots checkerseo crawler rules
L

This tool runs locally in your browser. Your input is not sent to a backend or paid API.

Robots.txt Tester

Check simple allow and disallow rules for a crawler and path.

How to use

How to use Robots.txt Tester

Follow this workflow to complete the task locally without leaving the page.

  1. 1

    Enter robots.txt rules, a user agent, and a URL path.

  2. 2

    Run the tool to check whether a robots.txt rule allows or blocks a path.

  3. 3

    Review the matched rule and crawl decision and any validation notes.

  4. 4

    Copy the result when it is ready.

Examples

Example inputs

Use these examples to understand common workflows and expected input formats.

Test a private path

Check whether a crawler should be blocked from a private URL segment.

User-agent: *
Disallow: /private/
Allow: /

FAQ

Robots.txt Tester FAQ

Answers to common questions about behavior, privacy, and practical usage.

Does Robots.txt Tester upload my input?

No. This tool runs locally in your browser and does not send your input to a backend or paid API.

What can I use Robots.txt Tester for?

Use it to check whether a robots.txt rule allows or blocks a path during development, SEO work, debugging, documentation, or publishing workflows.

Can I copy the output?

Yes. Generate the result and use the copy control to move it into your editor, docs, CMS, scripts, or test fixtures.

What happens if the input is invalid?

The tool shows a validation message when it can detect invalid or unsupported input so you can fix the value before using the output.