Sitemap Diff

Compare two XML sitemaps. See exactly which URLs were added, removed, or are common to both. Perfect for monitoring deploys, migrations, and competitor changes.

Sitemap diffs are one of those operations that sounds trivial — just compare two text files, right? — until you actually try to do it. Sitemap A is a single XML file with 3,000 URLs. Sitemap B is a sitemap index pointing at twelve child sitemaps, each with up to 50,000 URLs. URL canonicalisation differs (`/page` vs `/page/`, `http://` vs `https://`, `www` vs apex). Comparing them by-hand is a half-day of work; in this tool it's 5 seconds.

## Why this matters for production

Every CMS migration introduces silent sitemap changes. Every WordPress plugin install can add or remove a sitemap segment. Every Shopify theme update can change product URL structures. The damage is delayed: a page silently disappears from the sitemap, Google de-prioritises it over the next few weeks, traffic drops, and by the time someone notices the page has been deindexed it's been out of the sitemap for 6 weeks.

A pre/post-deploy sitemap diff catches all of this in 5 seconds. Save the URL list before deploy, save it after deploy, diff them. If the removed list contains anything that should still rank, your deploy has a bug.

## Competitor monitoring

The same tool, pointed at a competitor's public sitemap, tells you when they're publishing new content. Run it weekly: - Saturday's sitemap vs last Saturday's sitemap = their week's publishing output - New pages clustered around a theme = a new content campaign launching - Pages disappearing in bulk = a content prune or topic abandonment

This is one of the highest-signal, lowest-effort competitive intelligence inputs available, and most teams don't bother.

## URL canonicalisation — the hard part

Naïve diffing breaks on superficial URL differences: `/about` and `/about/` are the same page but text-different. We canonicalise both sides before comparing: lowercase host, strip trailing slashes (except root `/`), strip fragments, force https. This catches the 95% case. Edge cases (URL-encoded characters, default query parameters) may still cause noise — eyeball the diff if numbers look off.

## Sitemap index handling

We auto-expand sitemap indexes one level: if you point at `/sitemap-index.xml` we fetch each `<sitemap><loc>` child (up to 20) and aggregate URLs from all of them. This handles ~99% of real-world sitemap structures. We cap at 20 children and 10,000 URLs per side to keep responses fast and within edge-function timeout budgets.

## What we don't do (yet)

We don't check `lastmod` changes between matched URLs (same URL but content updated). We don't check sitemap-specific fields like `<priority>` or `<changefreq>` (Google ignores both, so neither do we). For deeper sitemap analytics — historical trends, lastmod drift detection, per-URL traffic mapping — use our paid monitoring tier.

What this tool checks

Why it matters

Every deploy, migration, or CMS upgrade silently changes your sitemap. Pages get added, removed, or have their URL structure shifted — often without anyone tracking the delta. A diff between yesterday's sitemap and today's catches removed-URL disasters (when 800 product pages quietly disappear) and added-URL surprises (when a CMS exports 12,000 tag pages it shouldn't). It's also the fastest way to monitor a competitor's content output.

How to fix what it finds

Methodology

We fetch each sitemap with a 15-second timeout. If the response contains `<sitemapindex>` we recursively fetch each child sitemap (max 20 children, depth 1) and aggregate all URLs. URLs are extracted via `<loc>` parsing and normalized: lowercase hostname, strip trailing slashes (except root), strip fragments, force https. Both URL sets are deduplicated. Diff is a simple set difference: `added = B \ A`, `removed = A \ B`, `common = A ∩ B`. We cap each diff list at 500 URLs in the response for readability; if you need the full diff, run a side-by-side download separately.

Frequently asked questions

Will it follow my sitemap index?

Yes — one level deep, up to 20 child sitemaps and 10,000 URLs per side. Larger sitemaps may be truncated.

Why do URLs that look identical show as different?

Most often it's trailing-slash differences (`/page` vs `/page/`) or http/https mismatch. We canonicalise both sides before diffing, but URL-encoded characters and query-string differences may still cause false positives.

Can I diff a live sitemap against an old one I have locally?

Not directly — both inputs must be URLs. As a workaround, upload your old sitemap to any public URL (S3, Gist, your own server) and point this tool at it.

Does this work for non-XML sitemaps?

No — we parse standard XML sitemap format (`<urlset>` or `<sitemapindex>`). For plain-text URL list sitemaps, this won't work.

How fresh is each side?

Live — we fetch both sitemaps in real time when you run the check. There's no caching.