Reject incomplete sitemap generation and recover after source failures

[tannerlinsley]

A failed or empty documentation manifest could produce a 200 sitemap with an entire library missing. Sitemap generation now fails explicitly with 503, no-store headers, and Retry-After instead of publishing that reduced result. Logs identify the failed library or catalog and record successful generation after recovery.

GitHub directory fallback now visits all depths and rejects a listed child directory that disappears. Cached trees with unvisited directories are rejected, and the manifest cache key changes so old depth-limited manifests are rebuilt. Local directory reads use the same complete traversal. Local-only one-millisecond file/tree caching was removed after the HTTP recovery test demonstrated that it kept serving an empty tree after the fixture was restored. Production GitHub cache behavior remains in place.

Stacked on #1264, which supplies the freshness metadata already used by sitemap entries.

Validation:

• Full pnpm test passed: 525 tests passed, 4 opt-in skips, TypeScript and lint clean. The commit hook repeated the final checks successfully.
• New remote fallback regression proves a missing child rejects without caching a partial tree, then recovers through an old incomplete cache with a deeply nested document intact.
• Real local HTTP integration passed with fixtures for all 18 sitemap-enabled documentation libraries. Removing the Start fixture and corrupting its frontmatter each produce an uncached 503; restoring each source immediately restores every expected library URL. Server logs name Start and show recovery.
• The integration uses the existing published/cached Charts catalog. It does not claim to simulate a catalog-service outage.
• sitemap-testing.md and scripts/prepare-sitemap-test.ts make the HTTP check reproducible.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 9ae87f52-f695-4c4e-9db1-c1cee0bb2596

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	tanstack-com	2a8a2b2	Oct 15 2026, 08:44 AM

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2a8a2b2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[tannerlinsley]

Updated this branch with the current docs-freshness parent and resolved the local-docs fallback conflict.

The integration test caught two cases worth keeping covered: an existing empty local docs directory must stay empty instead of falling back to remote docs, and malformed YAML must fail manifest generation instead of silently dropping metadata. Reference freshness handling also preserves that parse failure.

Verification on 2a8a2b2:

• Full pnpm test and commit hooks passed: types, lint, 562 unit tests, 2,269 chat tests and 20 desktop tests. Five unit tests and one desktop test were skipped.
• The isolated HTTP test covered all 18 sitemap-enabled docs libraries. Removing a source or corrupting its frontmatter returned 503 with no-store headers and Retry-After: 60. Restoring each source immediately restored the complete sitemap.
• Focused regressions cover missing-directory fallback, empty-directory recovery, malformed metadata and referenced metadata.

This still depends on #1264 and the required maintainer review. It only addresses part of #1153, which should stay open for the remaining canonical URL and deployed-page checks.