fix: keep what an in-memory output merge's precheck was granted, and overcommit a small shortfall

[msafonov]

Which issue does this PR close?

Closes #. (Joom fork, OOM diagnosis of night 10-17 -> 10-18, fix C)

Rationale for this change

parse_product_default on joom-1.1-12, stage 275 (JVM Sort 856 GiB + CometSort 20.7 GiB -> JVM SortMergeJoin LeftAnti -> UnsafeShuffleWriter): 7 task failures Additional allocation failed for ExternalSorterMerge[0] ... ExternalSorterMerge consumed 80.6 MB (can spill: false), the failing task's ShuffleExternalSorter holding 1544 MiB. The same stage had 0 failures on 1.1-9..1.1-11.

The output-merge precheck from #22 (output_merge_fits) probed the pool with try_grow and released the grant immediately; the workspace then kept only the merge headroom. Memory released to Spark during the merge, and the probe's grant itself, end up with the shuffle writer, which never spills for another consumer, so the merge's next few-KB growth gets 0 bytes.

What changes are included in this PR?

• reserve_output_merge (was output_merge_fits) returns the estimate and keeps the grant as a reservation; the in-memory merge takes it into its SpillWorkspace as a third parent and keeps max(headroom, needed - buffered) unused bytes reserved, so it grows into memory it already holds. keep_at_most now runs after the runs are registered, so the sorter's reservation is no longer released and grown back.
• SpillWorkspace::allow_overcommit: for the in-memory output merge only, a fallible growth the pool refuses is taken with the infallible grow while the total stays within max(16 MiB, 10% of the workspace); the Comet pool carries it as overcommit and repays from the next grant. Spill merges keep their own degradation; the bounded view stays strict.

How are these changes tested?

• New output_merge_keeps_what_its_precheck_was_granted (comet_memory_tests.rs): the precheck passes, then a second consumer takes the free pool and every released byte. On the base it fails with ResourcesExhausted("Failed to allocate additional 566.4 KB for ExternalSorterMerge[0] ..."), the production signature; with the fix it passes, also with the overcommit allowance set to 0 (the retained grant alone is enough).
• New refused_growth_overcommits_only_within_the_allowance (spill_workspace.rs).
• Local (aarch64 container): vendored physical-plan 1763 passed / 0 failed; workspace tests green (core 661, shuffle 190, spark-expr 972, jni-bridge 28, common 70); cargo fmt --check, cargo clippy --all-targets --workspace -D warnings clean.
• Cluster validation (parse_product_default for 2026-10-17 on the datasci-ab playground: 1.1-11, 1.1-12, fix, vanilla) follows.

🤖 Generated with Claude Code