Does this check internal links?

It checks same-host href links that it finds in the public static HTML, within the crawler limits. It does not crawl third-party destinations as part of this tool.

Does it crawl the entire site?

No. It explores at most two link levels, queues at most 50 new links per page and checks at most 200 unique URLs. A timeout can make the result partial sooner.

Some links show as "broken" but work in my browser — why?

Common causes include authentication, rate limiting, bot protection, network differences or a server that handles a HEAD request differently from a browser GET. Confirm important findings with a browser and the destination logs before changing a link.

How long does a crawl take?

It depends on the site's size, response times, and the number of links discovered. The result reports when a timeout or crawler limit makes the scan partial.

Does it follow nofollow links?

Yes — nofollow is an SEO directive, not a "do not check" directive.

What about links inside JavaScript-rendered content?

The crawler inspects the public HTML it receives. Links added only after client-side rendering may not be discovered.

Free Tool

Updated 22 September 2026

Broken Link Crawler

Crawl a public site to find failed internal links and report the URLs that need investigation.

By Deepak Shukla — Founder & CEO, Pearl Lemon

Enter a public site URL

What This Tool Checks

Same-host links found in the public HTML received from the starting URL
Up to two crawl levels, 50 newly queued links per page and 200 unique URLs
HTTP failures and connection errors from the crawler vantage point
Whether a page, link or time limit made the result partial

Why This Matters

Broken links send visitors and crawlers to dead ends. A crawler gives you a repeatable way to review public link failures after releases and content updates.

How to Fix Issues

1.Review each reported source page and link before changing it
2.For an internal broken link, update the source URL or restore the intended destination
3.For an external broken link, replace or remove it when there is no suitable live destination
4.Re-run the crawl after a significant content or template change

A broken-link crawl starts from a public URL and follows links within the limits of the checker. It is useful after a release, a migration, or a content update when you need a focused list of public failures to investigate.

What this tool does

Enter a public site URL. The crawler discovers same-host links from the static HTML it receives, then checks the URLs it collected. It reports the URL and the observed failure, rather than proving why a destination failed or where every link was placed in the site.

Where it shines

Use it for a focused quality check after a change. The public crawl queues no more than two link levels, 50 newly discovered links per page and 200 unique URLs, with a 15-second discovery budget and a 50-second overall budget. Read partial-result notices carefully: a limit or timeout means the result is not a complete inventory of the site.

Methodology

We begin at the URL you enter and extract href values from the static HTML returned for same-host pages. The crawler explores at most two levels, queues up to 50 new links per page and checks at most 200 unique URLs. It initially uses HTTP HEAD; blocked 401, 403 and 429 responses receive a limited browser-style GET recheck. HTTP failures and connection errors can be reported as broken. A result can be partial because of page, link or time limits, and a public check can be affected by authentication, rate limits, bot protection, network routing or a destination's handling of HEAD requests.

Frequently Asked Questions

Monitor this continuously with LemWatch

A one-off free broken link crawler shows you today. LemWatch re-runs the same checks around the clock and alerts you the moment something breaks — free for up to 10 sites, no card needed.