Skip to main content
Utilnivo

SEO workflow

Check if a site can be indexed

If crawlers are blocked or sitemaps are wrong, great content never ranks. This workflow builds and tests robots rules, prepares sitemaps, and checks redirects and broken links that waste crawl budget.

Category: SEO · Last updated: 2026-07-22

Start workflowCopy & customize

Step-by-step path

Steps (6 tools)

Generate and test robots.txt, build/validate sitemaps, and spot redirect or broken-link traps.

  1. Step 1: Draft robots.txt

    Tool: Robots.txt Generator

    Create allow/disallow rules for production crawlers.

    • On your device
    • No signup
    Open Robots.txt Generator

  2. Step 2: Test rules

    Tool: Robots.txt Tester

    Verify important paths are not accidentally blocked.

    • On your device
    • No signup
    Open Robots.txt Tester

  3. Step 3: Build sitemap

    Tool: Sitemap Generator

    Generate an XML sitemap of indexable URLs.

    • On your device
    • No signup
    Open Sitemap Generator

  4. Step 4: Validate sitemap

    Tool: Sitemap Validator

    Catch malformed XML or invalid loc entries.

    • On your device
    • No signup
    Open Sitemap Validator

  5. Step 5: Check redirects

    Tool: Redirect Checker

    Confirm old paths land on the correct final URLs.

    • On your device
    • No signup
    Open Redirect Checker

  6. Step 6: Find broken links

    Tool: Broken Link Checker

    Spot 404s that waste crawl budget and hurt UX.

    • On your device
    • No signup
    Open Broken Link Checker

When to use this workflow

  • A new site or staging→production cutover is imminent.
  • Pages are missing from Google despite being published.
  • You changed URL structures and need redirect confidence.

Tips

Tips for better results

  • Never disallow the URLs you want indexed.
  • Submit the final sitemap in Search Console after validation.
  • Fix chains and loops flagged by Redirect Checker before launch.

FAQ

Frequently asked questions

Can staging block indexing?

Yes—password gates, noindex, or robots disallow are common. Mirror production rules carefully at cutover.

Is a sitemap required?

Not strictly, but it helps discovery on large or new sites when linked from robots.txt and Search Console.

Keep going

Related workflows

Technical SEO health pass

SEO

Check HTTP status and headers, security headers, Core Web Vitals concepts, and redirects.

  1. Check status codes
  2. Inspect headers
  3. Review security headers
  4. Review CWV concepts
  5. Audit redirects
View workflow