Article 18.11.2026 20

XML Sitemap: How to Create & Check It (5,071 Sites, 2026)

We checked the XML sitemaps of 5,071 websites on November 15–16, 2026: 81.4% have a working one, 18.6% do not. Of the sites without one, most (11.9% of all) simply return a 404 at the sitemap address. This guide explains what an XML sitemap is, when you need one, how to create it with a CMS…

XML Sitemap: How to Create & Check It (5,071 Sites, 2026)

We checked the XML sitemaps of 5,071 websites on November 15–16, 2026: 81.4% have a working one, 18.6% do not. Of the sites without one, most (11.9% of all) simply return a 404 at the sitemap address. This guide explains what an XML sitemap is, when you need one, how to create it with a CMS plugin, a generator or our free tool, how to check it, how to tell Google about it, which lastmod rules matter and which limits apply, and then shows how often real sites get these things wrong. Caveat: our sample is non-random (sites from our search-results cache), so the numbers describe this sample, not the whole web.

Data: PromoPilot crawl of 6,000 sites, November 15–16, 2026; 661 sites blocked by a WAF, 178 unreachable and 90 with other errors excluded, 5,071 in the calculation

Key numbers

  • A working XML sitemap: 81.4% of 5,071 sites (4,126). None working: 18.6% (945), of which 404 at the sitemap address is 11.9% (605), an HTML page or other non-XML response (stub, soft 404) 3.4% (170), 403 for the crawler 1.3% (64).
  • Robots.txt declares a sitemap on 77.5% of sites. Among sites with a working sitemap, 90.0% declare it. On 4.2% of all sites (213) robots.txt points to a sitemap that does not work, and on 4.5% (228) the declared sitemap sits on another domain.
  • Of the working sitemaps, 65.1% are sitemap indexes (a sitemap of sitemaps) and 4.9% sit behind a redirect.
  • Lastmod: 28.6% of working sitemaps have none at all, 63.5% have it on almost every URL. Among the 2,946 sitemaps with a readable lastmod, the newest date is within 30 days in 76.2%, older than a year in 7.3%, and in the future in 1.2% (34 sitemaps).
  • Size: a single (non-index) sitemap has a median of 421 URLs, the 90th percentile is 7,945 URLs, and 18 are empty.
  • By CMS (generator meta tag): WordPress 97.8% have a working sitemap (597 sites), sites with no generator tag 78.7% (3,856), Drupal 78.7% (47), MediaWiki 7.1% (42).
  • Demand (Google Ads, monthly, worldwide): "sitemap generator" 60,500 searches a month (down 18% year over year), "xml sitemap generator" 12,100 (up 22%), "how to check sitemap of a website" 880, "sitemap lastmod" 320.

What an XML sitemap is and when you need one

An XML sitemap is a file that lists the URLs of your site that you want search engines to crawl, optionally with the date each page last changed. It is a hint, not a command: Google says a sitemap helps it discover pages, but does not promise that every listed URL will be crawled or indexed. It also does not replace good internal links. A page that no other page of your site links to (an orphan) can be found through a sitemap, which is one of its main uses.

Google's documentation says that a small site of about 500 pages or fewer, where all pages are reachable through internal links, may not need a sitemap. You usually do want one if:

  • the site is large (thousands of URLs) or changes often, so crawlers need a way to see what is new;
  • the site is new and has few external links, so crawlers have few ways in;
  • pages are poorly linked internally or you have many orphan pages;
  • you publish news, video or images and want to use the dedicated sitemap extensions;
  • you run several language versions and want to list them with hreflang annotations in the sitemap.

The demand data shows that people mostly want the file made for them: "sitemap generator" is searched 60,500 times a month, five times more than "xml sitemap generator" (12,100). The first fell 18% over the year while the second grew 22%, so people who already know the format are searching for it by name more often. Both are Google Ads monthly volumes for the whole world, rounded by Google, so treat the direction as reliable and the exact figures as approximate.

How to create an XML sitemap

Pick the method by how your site is built.

CMS plugin or built-in feature

If you use WordPress, you most likely already have one: WordPress has generated a core sitemap at /wp-sitemap.xml since version 5.5, and SEO plugins (Yoast SEO, Rank Math, All in One SEO) create their own, usually at /sitemap_index.xml or /sitemap.xml. Do not run two generators at once: check which URL your site actually serves and disable the duplicate. Shopify, Wix, Squarespace and Webflow generate the file automatically; you only need to find its address (normally /sitemap.xml) and submit it. Drupal and Joomla need a module or extension.

A generator

For a static site or a site without a plugin, a generator crawls your pages and writes the file. Our free Sitemap Generator does this: enter the address, get an XML file to upload to the site root. Desktop crawlers do the same for larger sites. Whatever you use, remember that a crawler finds only what is linked, so orphan pages will not appear in a generated file.

From a database or by script

For large custom sites, generate the file from your database (the list of published pages) on a schedule or on every publish. This is the only approach that keeps lastmod honest at scale. Make it exclude URLs you do not want indexed (noindex pages, redirects, 404s, URLs with tracking parameters, duplicate versions).

What the file looks like

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/guide/</loc>
    <lastmod>2026-11-12</lastmod>
  </url>
</urlset>

Each URL must be absolute (with https and the host), on the same site as the sitemap, and the file must be UTF-8 with special characters escaped. Only loc is required. Google ignores the priority and changefreq tags, so you can leave them out.

Limits: 50,000 URLs, 50 MB and the sitemap index

Table 1. Sitemap limits and what to do about them (Google Search Central and sitemaps.org)
LimitValueWhat to do
URLs in one sitemap file50,000Split into several files
Size of one file, uncompressed50 MBSplit; gzip helps transfer but the limit applies to the unpacked file
Sitemaps in one index file50,000Rarely reached

A sitemap index is a file that lists other sitemap files. It is the normal way to stay under the limits and to separate content types (pages, posts, products). In our crawl 65.1% of the working sitemaps are indexes, and the median index lists 7 child sitemaps. The other direction is common too: among single (non-index) sitemaps the median is 421 URLs and the 90th percentile 7,945, so the 50,000 limit is a concern for only a small part of sites. Our data does not show how many sites exceed it, so we do not give a number.

Share of 5,071 sites with a working XML sitemap and the reasons for failure
Does the site have a working XML sitemap? Share of 5,071 sites. PromoPilot crawl, November 15–16, 2026, non-random sample.

What we found on 5,071 sites

We requested the sitemap addresses most sites use, followed the robots.txt declaration, and parsed what came back. A sitemap counted as working if the response was valid XML with a urlset or sitemapindex. Table 2 shows how the 18.6% without one break down.

Table 2. XML sitemap status, 5,071 sites (PromoPilot crawl, November 15–16, 2026)
StatusSitesShare of 5,071
Working XML sitemap4,12681.4%
No working sitemap94518.6%
of which 404 at the sitemap address60511.9%
of which HTML or other non-XML (stub, soft 404)1703.4%
of which 403 (blocked for the crawler)641.3%
of which other failures1062.1%

Two of these deserve a comment. The HTML stub (3.4%) is a site that answers a sitemap URL with a normal page and a 200 status: a crawler sees a "successful" response, finds no XML and wastes the request. A 403 (1.3%) usually means a firewall treats crawlers as threats. Both are easy to miss if you only open the URL in a browser, which is why the check below uses the status code and the content type.

One more caution about the 18.6%: a missing sitemap is not an error for every site. A small, well-linked site can do without one, and we did not separate those from sites that need one.

How to tell Google where your sitemap is

There are two ways, and it is worth doing both.

  1. A line in robots.txt. Add Sitemap: https://example.com/sitemap.xml on its own line, with the full address. It can appear anywhere in the file and can be repeated for several sitemaps. Any crawler that reads robots.txt, not only Google, will find it. Our robots.txt checker shows what a crawler sees, and the robots.txt generator adds the line for you.
  2. Search Console. In the Sitemaps report of your property, enter the address and submit it. The report shows the date Google last read the file, the status and the number of discovered pages, and lists parsing errors. It is also where you will see "Couldn't fetch" if the address is wrong.

Google retired its old "ping" request for sitemaps in 2023, so you cannot nudge it by calling a URL: rely on robots.txt, Search Console and a correct lastmod.

In our data 77.5% of sites declare a sitemap in robots.txt, and among sites with a working sitemap the share is 90.0%. So 10% of working sitemaps are not announced in robots.txt and rely on crawlers guessing the address or on Search Console.

Robots.txt and sitemap: declaration, index files, redirects and broken pointers
Robots.txt and sitemap. The base of each bar is in its label. PromoPilot crawl, November 15–16, 2026, non-random sample.

Three declaration errors we saw

  • Robots.txt points to a sitemap that does not work: 4.2% of all sites (213). The line was left after a migration or a plugin change. Fix the line or the file.
  • The declared sitemap is on another domain: 4.5% (228). This is allowed if both sites are verified in the same Search Console account or the declaration is made in robots.txt, but it is often a leftover from a staging or old domain, so check that it is intentional.
  • The sitemap URL redirects: 4.9% of working sitemaps (201). Typical causes are http to https, or non-www to www. It usually works, but each redirect is an extra hop and a sign that the declared address is not the final one. Declare the final URL.

Lastmod: the one tag that matters, and how it goes wrong

Google's documentation says it uses lastmod when the value is consistently and verifiably accurate, for example when it matches the last significant change of the page. It ignores priority and changefreq. The rules that follow from this:

  • Set lastmod to the date of a real content change (text, price, availability), not to the date the file was generated or a footer was edited.
  • Use W3C Datetime format: 2026-11-12 or 2026-11-12T09:30:00+00:00.
  • Do not set every page to "today" on every build. If the date is always new, Google learns it is not reliable.
  • Never put a date in the future.
Table 3. Lastmod in working XML sitemaps (PromoPilot crawl, November 15–16, 2026)
MeasureSitesShareBase
No lastmod at all1,17928.6%4,126 working sitemaps
Lastmod on almost every URL2,62263.5%4,126 working sitemaps
Newest lastmod within 30 days2,24476.2%2,946 with a readable lastmod
Newest lastmod older than a year2147.3%2,946 with a readable lastmod
Lastmod in the future341.2%2,946 with a readable lastmod
Lastmod in working sitemaps: missing, fresh, stale, in the future
Lastmod in working sitemaps. The first two bars are shares of 4,126 sitemaps, the others of the 2,946 with a readable lastmod. PromoPilot crawl, November 15–16, 2026.

Read the table carefully. "Newest lastmod within 30 days" looks healthy (76.2%), but it says only that something on the site changed recently, not that the dates on individual pages are accurate: a site that stamps every URL with the build date also passes. We did not test accuracy page by page, so we cannot say how many of these dates Google would trust. What we can say is that more than a quarter of working sitemaps give no date at all, 7.3% have not changed their newest date for over a year, and 34 sitemaps contain dates in the future, which is a plain bug.

The demand side matches: "sitemap lastmod" is searched about 320 times a month worldwide, a small number, but it is the kind of query people type when something already looks wrong.

Sitemaps by CMS

We read the generator meta tag where a site publishes one. Table 4 shows the groups with at least 20 sites.

Table 4. Share of sites with a working sitemap, by CMS (generator meta tag; PromoPilot crawl, November 15–16, 2026)
CMS groupSitesWith a working sitemapShare
WordPress59758497.8%
Other CMS52946888.5%
No generator tag3,8563,03478.7%
Drupal473778.7%
MediaWiki4237.1%
Share of sites with a working sitemap by CMS
Share of sites with a working sitemap, by CMS. PromoPilot crawl, November 15–16, 2026, non-random sample.

WordPress stands out: 97.8% of the 597 sites have a working sitemap, which fits the fact that it generates one by default. MediaWiki is the opposite (3 of 42): it does not create a sitemap on its own, and a wiki needs an extension or a script. The largest group, sites that publish no generator tag, is 78.7%. Many of them use custom engines or hide the tag, so we cannot say which platforms they run, and the gap between 97.8% and 78.7% should be read as a property of this sample and not as a verdict on any platform.

How to check your sitemap

A sitemap can fail in ways you will not notice by opening it in a browser. A five-step check:

  1. Open the address and look at the response. You want status 200, a Content-Type of XML (application/xml or text/xml) and a body that starts with <urlset or <sitemapindex. A page with your design on it, even with status 200, is the stub we found on 3.4% of sites. Our redirect checker shows the status and the redirect chain.
  2. Parse it. The Sitemap Analyzer reads the file and its child sitemaps and reports what is inside, including the lastmod values, so you can compare your own site with Table 3.
  3. Sample the URLs. Open a handful: they should return 200, not redirect, and not be noindex. A sitemap full of redirected or deleted URLs teaches crawlers to distrust it.
  4. Check robots.txt. The Sitemap line must be present and point to the final address, and robots.txt must not block the sitemap URL or the pages in it.
  5. Look at Search Console. In the Sitemaps report, the status should be "Success" with a recent read date. For individual URLs, the site index checker and the Page indexing report show whether the pages in the file are indexed.

Common errors and how to fix them

Table 5. Common sitemap errors
ErrorWhy it mattersFix
404 at the sitemap addressCrawlers get nothing; this was the largest failure in our data (11.9%)Create the file or correct the URL in robots.txt and Search Console
HTML page instead of XMLStatus 200 hides the problemReturn real XML with an XML Content-Type; check caching and rewrite rules
Blocked by a firewall or login (403)Googlebot cannot read itAllow verified Googlebot; do not put the sitemap behind authentication
URLs that redirect, 404 or are noindexWastes crawl and weakens trustList only canonical, indexable 200 URLs
Wrong host or protocol (http, www, staging)URLs do not match the siteUse the exact canonical form of the URL
Lastmod always "today", or in the futureGoogle may stop trusting the dateUse the real change date
Over 50,000 URLs or 50 MBThe file is not read in fullSplit and use a sitemap index
robots.txt points to an old sitemap213 sites in our data (4.2%)Update the line after every migration

Methodology and limits

On November 15–16, 2026 we crawled 6,000 sites taken from our search-results cache. We excluded 661 sites that blocked the crawler with a WAF or similar protection, 178 that were unreachable and 90 with other errors, which leaves 5,071 sites in the calculation. For each site we requested robots.txt and the common sitemap addresses (up to three tries per site), followed redirects, and parsed the XML. Shares are shares of sites. The CMS comes from the generator meta tag, which many sites omit.

The limits you should keep in mind: the sample is not random and is built from sites that appear in search results, so it is not the whole web and the figures should not be generalized. Sites that blocked us are excluded, and that probably removes a part of the better-protected, larger sites. We checked that a sitemap exists and parses, not that every URL in it is valid or indexed, and we did not test the accuracy of individual lastmod dates. A missing sitemap is not an error for every site. The data contains no client projects and no domains. The aggregates are published as CSV under CC BY 4.0.

FAQ

Do I need an XML sitemap if my site is small?

Google says a site of about 500 pages or fewer that is fully linked internally may not need one. It still costs little to have one, and it helps for new sites, orphan pages and for monitoring in Search Console. A wrong sitemap is worse than none, so keep it accurate.

How do I create an XML sitemap for free?

If you use WordPress, Shopify, Wix or a similar platform, it is probably generated already: find the address and submit it. Otherwise use a generator such as our free Sitemap Generator and upload the file to the site root, then add the Sitemap line to robots.txt.

How do I check the sitemap of a website?

Open /sitemap.xml and /robots.txt, look for the Sitemap line, then confirm the status 200 and XML content, and parse the file with a tool such as our Sitemap Analyzer. Finish in the Sitemaps report in Search Console.

What should I put in lastmod?

The date of the last significant change to the page content, in W3C Datetime format such as 2026-11-12. Do not use the date the file was generated, and never a future date. Google uses it only if it is consistently accurate.

What is the sitemap limit?

50,000 URLs and 50 MB uncompressed per file. Above that, split into several files and list them in a sitemap index. In our data the median single sitemap has 421 URLs, so most sites are far below the limit.

Does a sitemap improve my rankings?

We cannot show that from this data and do not claim it. A sitemap helps crawlers discover and revisit pages, and Google does not promise crawling or indexing of listed URLs. It is a hygiene step, not a ranking lever.

Sources

How to cite

PromoPilot (2026). XML sitemap: how to create and check it, and what we found on 5,071 sites. Data November 15–16, 2026. CC BY 4.0: https://promopilot.link/uploads/blog/research/xml-sitemap-1811/xml-sitemap-5071-sites-2026-11.csv . Article: https://promopilot.link/en/blog/xml-sitemap-how-to-create-and-check-5071-sites-2026

Download the data

The aggregate tables (CSV, license CC BY 4.0): https://promopilot.link/uploads/blog/research/xml-sitemap-1811/xml-sitemap-5071-sites-2026-11.csv, plus the CMS breakdown https://promopilot.link/uploads/blog/research/xml-sitemap-1811/xml-sitemap-by-cms-2026-11.csv. It contains aggregates only, no projects, domains or client data.

Share:
Free · no sign-up

How many clicks is your site missing?

The SEO Radar shows in a minute which pages sit near the top of Google and what keeps them from moving up.

An estimate from positions, not a promise
New articles by email
Content