Add a CLDR person-name corpus and an agreement ratchet against its stated fields

AGENTS.md

@@ -28,7 +28,7 @@ Three committed contributor docs carry the parser's normative rules and their re
 
 **Counting claims.** A bare count in prose is either an assertion or a liability, keyed by who observes its staleness: asserted counts (a test holds the number) fail CI at change time — the useful kind; dated snapshots ("51 sites at spec time") cannot go stale; standing present-tense prose counts are the forbidden class — promote to an assertion, add a date, or state the invariant and let a test count. After changing how many times something runs, sweep for counts, not for the thing's name.
 
-**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary, be verified against rules.md examples, or -- for a view the gate cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. The classified summary covers the CONTRACT tier plus whatever radar diffs a rule classifies; a radar corpus's unmatched diffs are listed under UNCLASSIFIED (radar) and are not in it, so a claim quantified from the summary alone is silent about them. The gate compares the seven role fields, `_ambiguities` and -- since #484 -- `initials()`, under the `_initials` pseudo-field; so an initials-only change DOES show in the classified summary. But that pseudo-field sees only the names whose initials moved WHILE EVERY ROLE AND EVERY AMBIGUITY KIND STAYED PUT -- main()'s roles-identical guard keeps it out of any diff a role or a report is already in -- so a count taken from it is a FLOOR on initials movement, not the population: measured 2026-09-02 at 2.1.0 → tree on the v2 surface, 83 of the 1120 compared ENTRIES (1116 distinct names; seven entries carry a declared order rather than the default, and three strings are compared under more than one) changed their `initials()` string and only 28 were visible under `_initials`, the other 55 having moved a role as well. A bullet about how many names' initials changed still needs the recompute recipe. `capitalized()` and any other render view stay invisible to the gate (decisions.md#R4, #R3), and for those the first two sources still cannot reach a claim: a gate run is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. The same standard binds decisions entries and PR bodies: a claim that a change restores the released reading is measured over the whole moved population against the wheel, never spot-checked (#625 claimed its 774 moves landed on the 1.4.0 and 2.3.0 readings from three names; measured, the tree matched 2.3.0 on 0 of 258 moved texts, master on 152), and a population size is re-counted from its recipe, never copied from another entry (the same entry carried #624's 137,810 for a grid of 395,148). Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline, and since 2026-10-10 tests/v2/test_doc_citations.py checks them as it checks code. A ledger comment is copied from ledger to ledger and reworded by nothing: measured 2026-10-10 by running the check as it now stands over the tree before it (8e6e524a, master after #631), 27 quotes were stale and 5 colon citations quoted nothing, beyond the five #631's review found by hand -- most in a shape the colon-only check could not read (`rules.md#X -- "..."`, `rules.md#X's Accepted clause ("...")`, a second quote chained by "and"). A rule's statement and its `Accepted:` clauses are quotable, its example lines are not; `rules.md#H` names the H section, whose Background is quotable; a decisions.md citation may paraphrase, but what it quotes must be verbatim; and a reference with no quote must still name something that exists. The check's reach is a quote within four words of the ID or chained to one that is: a quote further off reads exactly like a name written in double quotes, so it is NOT checked, and the second review of #632 found 14 such quotes stale by hand. Keep a quote beside its ID. Quotes of AGENTS.md itself are not checked at all. Free prose is for ledger mechanics only (owned by tools/differential/README.md).
+**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary, be verified against rules.md examples, or -- for a view the gate cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. The classified summary covers the CONTRACT tier plus whatever radar diffs a rule classifies; a radar corpus's unmatched diffs are listed under UNCLASSIFIED (radar) and are not in it, so a claim quantified from the summary alone is silent about them. The reverse holds for `corpus_cldr.jsonl` (2026-10-12): its fictional renderings DO enter the summary wherever a rule classifies them, so count them out of a release figure (tools/differential/README.md, the CLDR section). The gate compares the seven role fields, `_ambiguities` and -- since #484 -- `initials()`, under the `_initials` pseudo-field; so an initials-only change DOES show in the classified summary. But that pseudo-field sees only the names whose initials moved WHILE EVERY ROLE AND EVERY AMBIGUITY KIND STAYED PUT -- main()'s roles-identical guard keeps it out of any diff a role or a report is already in -- so a count taken from it is a FLOOR on initials movement, not the population: measured 2026-09-02 at 2.1.0 → tree on the v2 surface, 83 of the 1120 compared ENTRIES (1116 distinct names; seven entries carry a declared order rather than the default, and three strings are compared under more than one) changed their `initials()` string and only 28 were visible under `_initials`, the other 55 having moved a role as well. A bullet about how many names' initials changed still needs the recompute recipe. `capitalized()` and any other render view stay invisible to the gate (decisions.md#R4, #R3), and for those the first two sources still cannot reach a claim: a gate run is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. The same standard binds decisions entries and PR bodies: a claim that a change restores the released reading is measured over the whole moved population against the wheel, never spot-checked (#625 claimed its 774 moves landed on the 1.4.0 and 2.3.0 readings from three names; measured, the tree matched 2.3.0 on 0 of 258 moved texts, master on 152), and a population size is re-counted from its recipe, never copied from another entry (the same entry carried #624's 137,810 for a grid of 395,148). Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline, and since 2026-10-10 tests/v2/test_doc_citations.py checks them as it checks code. A ledger comment is copied from ledger to ledger and reworded by nothing: measured 2026-10-10 by running the check as it now stands over the tree before it (8e6e524a, master after #631), 27 quotes were stale and 5 colon citations quoted nothing, beyond the five #631's review found by hand -- most in a shape the colon-only check could not read (`rules.md#X -- "..."`, `rules.md#X's Accepted clause ("...")`, a second quote chained by "and"). A rule's statement and its `Accepted:` clauses are quotable, its example lines are not; `rules.md#H` names the H section, whose Background is quotable; a decisions.md citation may paraphrase, but what it quotes must be verbatim; and a reference with no quote must still name something that exists. The check's reach is a quote within four words of the ID or chained to one that is: a quote further off reads exactly like a name written in double quotes, so it is NOT checked, and the second review of #632 found 14 such quotes stale by hand. Keep a quote beside its ID. Quotes of AGENTS.md itself are not checked at all. Free prose is for ledger mechanics only (owned by tools/differential/README.md).
 
 **Writing the user docs (docs/*.rst) has its own AGENTS.md too.** `docs/AGENTS.md` carries the style the docs are written in — tables as indexes, subheadings per task, measured claims, recipe doctests — distilled from PR #588. It loads the same way as the docs/design/ one below; if your tool does not do nested discovery, read it before editing a `.rst` file under docs/.
 
@@ -372,7 +372,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_
 
 **Adding a flag-gated post-parse transform** (reorder/adjust) — add a `Constants` boolean (default `False`), implement a `handle_*()` method, and call it in `post_process()` after `handle_firstnames()` and before `handle_capitalization()`, gated on the flag. Default-off keeps existing parses byte-for-byte unchanged. Two shipped examples: `patronymic_name_order` gates both `handle_east_slavic_patronymic_name_order()` (#85) and `handle_turkic_patronymic_name_order()` (#185) — one flag driving two independent handlers, added in the same `post_process()` slot; `middle_name_as_last` gates `handle_middle_name_as_last()` (#133), which folds `middle_list` into `last_list`.
 
-**Validating a new parsing rule** — before implementing, simulate it in a throwaway script against `TEST_NAMES` (plus a few target-language examples) to catch regressions/false-positives early. E.g. this surfaced the `last_base` `do`/`st`/`mc` empties and the patronymic `"David Michael Abramovich"` false-reorder.
+**Validating a new parsing rule** — before implementing, simulate it in a throwaway script against `TEST_NAMES` (plus a few target-language examples) to catch regressions/false-positives early. E.g. this surfaced the `last_base` `do`/`st`/`mc` empties and the patronymic `"David Michael Abramovich"` false-reorder. After the change, `uv run python tools/differential/cldr_agreement.py` reports how the tree's agreement with CLDR's stated name fields moved; `tests/v2/test_cldr_agreement.py` fails on any move, and `tools/differential/cldr_agreement.json` doubles as a measured list of names the parser and CLDR read differently.
 
 ## Gotchas
 

tests/v2/_differential_fixtures.py

@@ -40,10 +40,25 @@ def _entry_name(raw: object) -> str:
     return raw["name"]
 
 
-#: Every name the harness classifies, deduplicated. The ledgers exist
-#: to explain diffs on THESE strings and no others, so "what does this
-#: rule claim?" is answerable here without parsing anything -- a plain
-#: regex search, no baseline wheel, no network.
+#: Corpus files the harness compares but whose names this population
+#: leaves out. corpus_cldr.jsonl (2026-10-12): about 1,900 fictional
+#: CLDR renderings whose ground truth the agreement ratchet in
+#: test_cldr_agreement.py enforces. Counted here they moved the
+#: recorded reach of fourteen 1.4.0-ledger rules at once (the comma
+#: rules 461 -> 771 names, feat(#269) 6 -> 196), and the decision was
+#: to keep the recorded claims about the names the ledgers were written
+#: for. The harness still classifies CLDR diffs against the ledger on
+#: the radar tier; only this population does not measure them. compare.py
+#: holds the same set as its own _UNMEASURED_CORPORA, which keeps these
+#: names out of contest adjudication too; test_differential.py holds the
+#: two equal.
+_UNMEASURED_CORPORA = frozenset({"corpus_cldr.jsonl"})
+
+#: Every name the harness classifies, deduplicated, except the files in
+#: _UNMEASURED_CORPORA. The ledgers exist to explain diffs on THESE
+#: strings, so "what does this rule claim?" is answerable here without
+#: parsing anything -- a plain regex search, no baseline wheel, no
+#: network.
 #:
 #: This is what test_ledger_guards.py's corpus checks read, and why
 #: they hold where four rounds of syntactic ones did not. Depth-0 pipes, nesting
@@ -53,6 +68,7 @@ def _entry_name(raw: object) -> str:
 _CORPUS_NAMES = sorted({
     _entry_name(json.loads(line))
     for path in sorted(_TOOLS.glob("corpus*.jsonl"))
+    if path.name not in _UNMEASURED_CORPORA
     for line in path.read_text(encoding="utf-8").splitlines() if line.strip()})
 
 

tests/v2/conftest.py

@@ -28,9 +28,19 @@ def differential_corpus() -> list[str]:
     tools/ is still not imported: it is a standalone uv-run script tree
     rather than a package, and the one-JSON-string-per-line format is
     cheaper to read than to make importable.
+
+    corpus_cldr.jsonl is left out, for now and for a measured reason.
+    Joining it on 2026-10-12 took test_properties.py from 34.5 s to
+    39.6 s (py3.11, local) -- affordable -- but four of its Slavic
+    renderings fail test_a_decomposed_name_renders_as_its_composed_twin:
+    decomposed `inž.` is not read as a period abbreviation, so
+    `Janez Novak, univ. dipl. inž.` reverses (#652). Re-include the
+    file when that is fixed. The harness and the CLDR agreement ratchet
+    still parse these names.
     """
     corpus_dir = Path(__file__).parents[2] / "tools" / "differential"
-    paths = sorted(corpus_dir.glob("corpus*.jsonl"))
+    paths = sorted(p for p in corpus_dir.glob("corpus*.jsonl")
+                   if p.name != "corpus_cldr.jsonl")
     assert paths, f"no corpus*.jsonl in {corpus_dir}"
     names = [_entry_name(json.loads(line))
              for path in paths