What it does
A sitemap should be a clean list of your live pages. In practice it drifts: pages get deleted, URLs change, plugins add archives nobody asked for. This tool shows you what's actually in it.
Give it a sitemap URL or just a domain. With a domain it reads the Sitemap lines in robots.txt, or falls back to /sitemap.xml. It follows sitemap index files, counts the URLs, and checks whether the first 20 answer 200.
How to use it
- 1Enter an address in Sitemap URL, or a domain to find it, then press Read the sitemap.
- 2Read the table of files: whether each is a sitemap index or a URL list, how many entries it holds, or the error if it couldn't be read.
- 3Check the results under it: the total URL count, whether every URL is on this site, and how many of the first 20 don't answer 200.
- 4Copy the URL list, or Download it as sitemap-urls.txt.
Reading the results
A sitemap index is a sitemap of sitemaps, common on large sites and CMS plugins. The tool reads up to 8 files and says if more were left unread. URLs on another domain are ignored by search engines; they usually come from a staging host or an old domain in a config file.
Each sampled URL is followed through up to 5 redirects. Anything that doesn't end on a 200 is listed with where it ended. A URL that redirects and then answers 200 isn't flagged, so paste the list into the bulk status checker to find redirects.
Common sitemap problems
Fix whatever generates the sitemap rather than editing the file by hand, or these come back on the next build.
- Deleted pages that now return 404 or 410.
- Old addresses that redirect instead of the final ones.
- Both http and https, or www and non-www, versions of a page.
- Pages marked noindex, or with a canonical pointing elsewhere.
Questions
How do I find a site's sitemap?
Look for a Sitemap: line in /robots.txt, or try /sitemap.xml. This tool does both when you enter just a domain.
What is a sitemap index?
A file that lists other sitemap files instead of pages. Sites need one past 50,000 URLs or 50 MB in a single file, and many use one to group pages by type.
Why does a URL fail here but work in my browser?
Some servers refuse automated requests with 403 or 429, and some answer too slowly. Open the URL yourself before you remove it.
Does it read compressed sitemaps?
No. Gzipped .xml.gz files aren't read, and each file is read up to 5 MB. Enter the uncompressed address instead. Plain-text sitemaps with one URL per line work.