Umbodi
Get started
Receipts · sitemap_url_not_indexable

How does an SEO agent safely remove unreadable pages from a sitemap?

Umbodi removes sitemap entries whose pages serve nothing a crawler can read, only when the owner has opted in to crawler-policy changes, publishes it as a pull request, and checks the entries are gone.

Not fixed by Umbodi on its own site — it cannot be planted there

What it is, and why it matters

A sitemap entry that serves an error or an empty page sends crawlers to a dead end and spends their attention on a page that cannot be indexed.

How it is detected

Fires for each sitemap URL that serves nothing a crawler can read — an empty or script-only page, an HTTP error, or no site at all. Today only a site audit started from the app raises it; the nightly run does not yet. Severity: warn or error.

How it is repaired

The agent removes those URLs from the sitemap and refuses if that would leave it empty. It is one of two opposite remedies — the other is to fix the page — so it acts only with the owner's crawler-policy opt-in.

The change reaches the site as a pull request that adds or edits a site file at the repository's web root, opened by the Umbodi GitHub App. Generated blocks sit inside fences the agent owns, so the site's hand-written lines are never rewritten.

When the agent may act with nobody present

The value is derived from the site's own data, so the change is stamped mechanical and no score bar applies. It still applies only when the owner has armed unattended acting and set this category's autonomy dial to automatic; merging needs a separate opt-in.

Because this changes what crawlers may read, there is one more gate: the owner must have opted the project in to crawler-policy changes. Without that, the agent proposes and waits.

One proposal in ten is held back as a control group — chosen by a fixed hash per project, not by hand — and is never applied unattended, so the effect of the fixes can be measured against the ones not made.

How the repair is verified

After the pull request merges and the production deploy appears, the agent fetches the live file and compares it with the merge's own diff: every line it added must be there, and a change that only removed lines must have them gone. If not, the file is re-read up to five times, about two and a half hours, and then the agent reverts its own merge.

When it stops instead

It stops, and says why, when the live file is generated by the site's own code rather than served from the repository — a pull request would change nothing — when the file in the repository already says the same, when a robots.txt change needs a policy the owner has not chosen, or under the same merge and deploy rules as page changes. It never reverts on a file it cannot fetch.

The receipt from this site

Our site has nowhere this defect can be planted without breaking something real, so there is no live receipt. The repair path is proven in the test suite only. The reason, as the code states it:

plantable in principle and not built: it needs a new sitemap edit that ADDS a url serving nothing (the existing sitemap_lastmod target only ages an entry), and the drill then advertises a dead page to crawlers for up to 52 hours. Measured 2026-10-07: the remedy and the plant are inverses — the agent dropping the url restores exactly what scripts/build-sitemap.mjs produces — so this is the one of the three that a later session could reasonably add. It is listed rather than attempted because a drill that lies to a crawler about a page should be a deliberate decision, not a side effect of finishing a list.

Last verified 2026-10-11 08:43 UTC

What this proves, and what it does not

Only what the test suite proves: the route from finding to verified repair works against fixtures. Nothing on this page claims it has happened live.

It does not prove a ranking or citation change. Umbodi never infers one from a fix: rank is measured separately, on Google, and reported as measured. One receipt also shows the path works on this site, not that it works on every site.

Questions about this defect.

How does an SEO agent safely remove unreadable pages from a sitemap?

Umbodi removes sitemap entries whose pages serve nothing a crawler can read, only when the owner has opted in to crawler-policy changes, publishes it as a pull request, and checks the entries are gone.

Has Umbodi fixed this on its own site?

Not live. Our site has nowhere it can be planted without breaking something real, so the repair is proven in the test suite only, and this page says why.

Every defect type, with its receipt.

The receipts page lists every defect type the agent repairs on its own, and which of them it has repaired live on this site.

Umbodi acts on your mandate — an autonomous agent for search and AI-answer visibility. It finds the work, ships the work, and measures what happened.
Umbodi™ is a trademark of I-Novanet Software Ltd. Application accepted in Mauritius (MU/M/2026/44210).