noindex checker
Check whether Google can actually index your page. Detects noindex, canonical conflicts, X-Robots-Tag headers, soft 404s, and crawl blockers.
By Deepak Shukla — Founder & CEO, Pearl Lemon
Part of SEO monitoring — the full answer, the facts and the alternatives in one place.
What This Tool Checks
- Meta robots tag (noindex / nofollow)
- X-Robots-Tag HTTP header
- Canonical URL (and whether it points away from this page)
- HTTP status code (200 / 3xx / 4xx / 5xx)
- Soft 404 detection (200 status with "not found" content)
- Redirect chains preventing index
Why This Matters
A page that isn't indexable cannot rank — no matter how good the content is. The most common SEO disaster is a single accidental `noindex` in a CMS template silently de-indexing thousands of pages. This checker tells you in 5 seconds whether your page can rank at all.
How to Fix Issues
- 1.Remove `<meta name="robots" content="noindex">` from pages you want indexed
- 2.Strip `X-Robots-Tag: noindex` from your server headers
- 3.Make sure the canonical points to itself (or a deliberate target)
- 4.Replace soft 404s with real 404 status codes
- 5.Eliminate redirect chains; one hop max
The single most common SEO disaster is a page that nobody realises has been quietly de-indexed for months. A single noindex left in a CMS template, a misconfigured CDN header, a copy-pasted canonical pointing at staging — any one of these can wipe a page out of Google's index without warning. This checker is the fastest way to confirm a page can actually rank.
The five ways Google decides not to index a page
- Meta robots noindex — the most common, a
<meta name="robots" content="noindex">in the head - X-Robots-Tag header — same effect, but sent via HTTP header (often by CDN or WAF rules)
- Canonical pointing away — a
<link rel="canonical">pointing at a different URL tells Google to index the canonical instead - HTTP status — anything other than 200 (with the exception of follow-able 3xx) signals "don't index this URL"
- Soft 404 — page returns 200 but the content is "not found" — Google detects and de-indexes
This tool checks all five in one request.
The canonical trap
Canonicals are a preference signal, not a guarantee. A self-referential canonical is recommended for the preferred page. A canonical pointing to a genuinely duplicate URL can help search engines consolidate signals; a mistaken target, such as a staging URL, creates conflicting evidence that should be corrected.
X-Robots-Tag — the invisible noindex
Because it's an HTTP header, X-Robots-Tag is invisible in "view source" — developers can't see it without inspecting the network panel or using a tool like this. CDNs (Cloudflare, Vercel, Netlify) often add X-Robots-Tag: noindex on preview or staging deployments. If that header survives the promotion to production, you have a silent disaster.
What about robots.txt?
robots.txt blocks crawling, not indexing — if a URL is linked from elsewhere, Google can still index it without ever crawling it (showing a SERP entry with no description). To prevent indexing, use noindex, not robots.txt. To prevent crawling AND indexing, allow crawling so Google sees the noindex, then add the noindex.
Soft 404 detection
Google's definition of a soft 404 is fuzzy and undocumented — they look at content length, presence of "not found" strings, structural similarity to your real 404 page, and traffic patterns. Our heuristic catches the easy cases (very short response, telltale phrases) but Google's detection is more aggressive. The safe pattern: serve a real 404 status for missing pages, always.
Methodology
We fetch the URL with a Googlebot-compatible User-Agent and a 15-second timeout, following redirects up to 5 hops. We capture the final HTTP status code, every relevant header (X-Robots-Tag, Cache-Control, Content-Type), and the full HTML body. From the HTML we extract <meta name="robots">, <link rel="canonical">, the <title>, and structural signals for soft-404 detection. Each indexability blocker is reported separately so you can fix them one at a time. We do NOT execute JavaScript — this matches what Googlebot sees on its first pass and surfaces issues that JS-rendered checks would miss.
Frequently Asked Questions
This tool also answers
How do I check a redirect chain?
Enter the starting URL. The checker follows every hop and shows each status code in order, so you can see whether a link passes through two or three redirects before landing — each extra hop wastes crawl budget and leaks a little link equity.
How do I check the canonical URL of a page online?
The result includes the rel="canonical" value found in the HTML and in the HTTP headers, plus a warning when the two disagree or when the canonical points at a different page than the one you requested.
How do I check which pages Google has indexed?
This tool tells you whether a specific URL is technically indexable — no noindex, no blocked robots rule, no conflicting canonical. For the full list of what Google actually holds, pair it with the Index Coverage report in Search Console.
Related Free Tools
Broken Links
Crawl your site and find every 404, 5xx, redirect chain, and dead external link. See exact URLs, source pages, and HTTP status codes.
Sitemap Checker
Inspect your XML sitemap. Find it, check its basic sitemap structure, count URLs, detect sitemap indexes, and flag suspicious or broken sub-sitemaps.
Content Freshness
Find which pages on your site haven't been updated in months. See last-modified dates, age in days, and which pages need a refresh to keep ranking.
HTTP Status Code Reference
Understanding HTTP status codes helps you diagnose issues found by this tool: