XML Sitemaps: Creation, Optimization, and Error Handling

When a Sitemap Matters

Google’s sitemaps overview says “in most cases, your site will benefit from having a sitemap”, and then lists when you might need one and when you might not.

You might need a sitemap if	You might not need one if
The site is large, so some pages may not be linked from any other page	The site is "small", about 500 pages or fewer that need to be in search results
The site is new and has few external links	The site is "comprehensively linked internally" from the home page
The site has a lot of video or images, or appears in Google News	There aren't many media files or news pages you want shown in search

A sitemap is not a substitute for internal links. A page that only exists in the sitemap is an orphan; link it from relevant pages as well, as covered in internal linking strategy.

Building the File

Google’s sitemap build guide sets the limits and format rules:

• “All formats limit a single sitemap to 50MB (uncompressed) or 50,000 URLs.” Larger sets must be split, optionally under a sitemap index.
• “The sitemap file must be UTF-8 encoded.”
• “Use fully-qualified, absolute URLs in your sitemaps. Google will attempt to crawl your URLs exactly as listed.”
• A sitemap not submitted through Search Console “affects only descendants of the parent directory”, which is why Google recommends the site root.
• “As with all XML files, all tag values must be entity escaped.”

A minimal file:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/services/roof-repair/</loc>
    <lastmod>2026-10-12</lastmod>
  </url>
  <url>
    <loc>https://www.example.com/guides/choosing-roofing-materials/</loc>
    <lastmod>2026-09-30</lastmod>
  </url>
</urlset>

There is no priority or changefreq in it, on purpose. The build guide says: “Google ignores <priority> and <changefreq> values”. Google’s June 2023 announcement on sitemaps was blunter: “Google still doesn’t use the changefreq or priority elements at all.”

For large sites, a sitemap index lists the individual sitemaps:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://www.example.com/sitemaps/products.xml</loc>
  </sitemap>
  <sitemap>
    <loc>https://www.example.com/sitemaps/guides.xml</loc>
  </sitemap>
</sitemapindex>

Splitting by content type has a reporting benefit Google mentions: submitting multiple sitemaps “may be useful if you want to track the search performance of each individual sitemap in Search Console.”

lastmod: Useful Only When It’s True

Google uses lastmod for crawl scheduling, with a condition. The build guide: “Google uses the <lastmod> value if it’s consistently and verifiably (for example by comparing to the last modification of the page) accurate”. It defines what counts: the value “should reflect the date and time of the last significant update to the page. For example, an update to the main content, the structured data, or links on the page is generally considered significant, however an update to the copyright date is not.”

The 2023 announcement explains what happens when the dates drift: “if your page changed 7 years ago, but you’re telling us in the lastmod element that it changed yesterday, eventually we’re not going to believe you anymore when it comes to the last modified date of your pages.”

It also allows a sensible exception: “You can use a lastmod element for all the pages in your sitemap, or just the ones you’re confident about.” Aggregation pages such as the homepage or category pages, whose software can’t tell when they last changed, can leave it out.

Two things produce untrustworthy dates: a generator that stamps every URL with the build time, and a theme change that updates every page’s modified date. Check a sample of URLs against the date their main content actually changed; if they don’t match, fix the generator before adding more URLs.

What to Include and What to Leave Out

Google’s guidance is to “Include the URLs in your sitemap that you want to see in Google’s search results.” In practice:

• Include: canonical URLs that return 200 and that you want indexed.
• Leave out: redirecting URLs, 404 and 410 URLs, pages with noindex, non-canonical duplicates, URLs blocked by robots.txt, and utility pages such as cart, login and internal search results.

Platforms generate URLs you may not want listed, such as tag archives, author archives or attachment pages. Check what your CMS or plugin actually outputs, and remove the types you don’t want indexed from the sitemap at the same time as you decide how to handle them on the site.

Image, Video and News Sitemaps

• Images. Google’s image sitemap documentation lists two required tags, <image:image> and <image:loc>, declared with the http://www.google.com/schemas/sitemap-image/1.1 namespace, and allows “up to 1,000 <image:image> tags” per URL. Older guides also show caption and title tags; they are not in Google’s current documentation. Put image descriptions in alt text and on the page, as covered in image SEO.
• Video. Video sitemaps can add details about videos on your pages; how they fit with VideoObject markup is covered in video schema.
• News. Google’s news sitemap documentation says to “Only include recent URLs for articles that were created in the last two days”, and a news sitemap may have “up to 1,000 news:news tags.” News visibility more broadly is in Top Stories optimization.

An image sitemap with the required namespace:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:image="http://www.google.com/schemas/sitemap-image/1.1">
  <url>
    <loc>https://www.example.com/products/oak-desk/</loc>
    <image:image>
      <image:loc>https://www.example.com/images/oak-desk-front.jpg</image:loc>
    </image:image>
  </url>
</urlset>

Submitting and Notifying

Two methods reach Google: submit the sitemap in Search Console’s Sitemaps report, or list it in robots.txt with a Sitemap: line. The ping endpoint is gone; Google’s 2023 announcement said it would “stop functioning in 6 months” and that requests to it “will result in a 404 error”, adding that existing code using it “will not cause problems for Google Search”, but it “will also not do anything useful”.

Resubmitting is for real changes. The Sitemaps report notes that if a sitemap has no critical updates, you can “let Google follow its regular crawling schedule.”

Other search engines publish their own submission methods. The IndexNow protocol is one way to notify participating engines of changed URLs; its documentation notes that an HTTP 200 response “only indicates that the search engine has received your URL.” Check each engine’s webmaster documentation for what it supports.

Errors and How to Read Them

• Can’t be read. Invalid XML, the wrong namespace, a non-UTF-8 file or a file over the size limits. Validate the XML before submitting.
• Couldn’t fetch. The sitemap URL returns an error or is blocked. Google’s Sitemaps report says that if a fetch fails, Google “will retry for a few days, and then stop if the sitemap continues to be unavailable or has critical errors”, so fix it and resubmit.
• URLs that contradict the sitemap. Listed URLs that redirect, return errors, carry noindex or are blocked by robots.txt. They don’t break the file, but they send conflicting signals; clean them out at the generator.
• Listed but not indexed. A URL in the sitemap can still be “Crawled – currently not indexed” in the Page indexing report. That is an indexing decision about the page, not a sitemap error; see how Google crawls and indexes websites.

Keeping It Healthy

Generate the sitemap from the same source of truth as the site, so URLs appear when pages publish and disappear when they’re removed or redirected. Regenerate when content changes rather than on a fixed schedule that rewrites every date. Compare the sitemap with a crawl of the site now and then: URLs in the sitemap but not linked anywhere are orphans; important pages linked on the site but missing from the sitemap point to a generator gap. Site structure decisions that make all of this easier are covered in site architecture and URL structure.

When a URL from your sitemap shows as “Crawled – currently not indexed”, work on the page rather than the sitemap: Google has already found and crawled it, so resubmitting the file won’t change a decision about the page itself.