googlesql: Spanner schemas, arrays, and a function seed from Spanner

[kyleconroy]

With zetajones v0.2.0 parsing Spanner's DDL, this fills the GoogleSQL engine's remaining gaps for Spanner. A realistic Spanner schema and query set that used to fail or come back as any now generates fully typed code. The PR has four commits, one per area.

1. Schema statements

• ALTER TABLE used to be dropped as unsupported, so columns it added, dropped or retyped never reached the catalog. It now converts ADD/DROP COLUMN, Spanner's ALTER COLUMN (which restates type and nullability), ALTER COLUMN … DROP NOT NULL, RENAME COLUMN and RENAME TO.
• RENAME TABLE, CREATE [OR REPLACE] VIEW and DROP VIEW now convert.
• A HIDDEN column is left out of SELECT *, as Spanner does.

2. Arrays and literals

• Converter: typed literals (DATE, TIMESTAMP, NUMERIC, JSON, bytes), array constructors, INTERVAL, the OFFSET/ORDINAL/SAFE_* subscripts, and x IN UNNEST(arr), which becomes x = ANY(arr).
• Core: a subscript is the array's nullable element with an integer index, and ARRAY(subquery) is an array, no longer its first value. x = ANY(array) types a placeholder as an array of x's type, which is what PostgreSQL reports; sqlc.slice() still gets the element. unnest(anyarray) → anyelement resolves to the element, and a function in FROM takes its resolved type, with its alias naming the column.

3. Functions and operators

• functions.jsonl had only count. It now has 291 signatures across aggregates, math, strings, dates and timestamps, JSON, arrays, hashing, search and vector distance. operators.jsonl has 83. Spanner has no window functions, so none are listed.
• GoogleSQL now propagates NULL through function arguments. Functions that return NULL from non-null input (JSON_VALUE, REGEXP_EXTRACT, the SAFE_* functions, …) are marked nullable.
• The converter maps COALESCE/IFNULL/NULLIF/IF/GREATEST/LEAST onto the nodes the core types them by, makes SAFE.f(...) nullable, passes date parts (DAY, WEEK(MONDAY)) as strings rather than columns, and converts EXTRACT.
• Core: an operator a dialect lists now replaces the one its arithmetic rule generates for the same operands, which is how INT64 / INT64 is FLOAT64. A placeholder passed for a polymorphic parameter takes its type from the other arguments, so ARRAY_INCLUDES(arr, @v) types @v as the element.

4. goldeneye generates the library

Spanner keeps no catalog of its functions, so goldeneye now reads them by asking.

• goldeneye/spanner/library.jsonl lists the signatures to try. Each is compiled in PLAN mode against a probe database, and the ones Spanner accepts are written to functions.jsonl with the type Spanner reports. A polymorphic signature is tried with INT64 and STRING, and its result must hold for both.
• Calls a select list can't make are compiled the way they can be: SEARCH in a WHERE clause, SCORE over a search index, PENDING_COMMIT_TIMESTAMP in an INSERT.
• operators.jsonl is every binary operator Spanner accepts over the operands tried. types.jsonl is the types the dialect lists, each checked by a cast. The GoogleSQL types only BigQuery has (bignumeric, datetime, time, geography, range, map, …) are kept as they are, since the engine is shared with BigQuery.
• Nullability is the one thing Spanner doesn't report, so library.jsonl states it.
• The generated files hold exactly the records the seed in commit 3 had. TestDialect now checks all four files against Spanner.

Effect on other engines

The core changes reach every engine that goes through the analysis core. One existing golden changes: in analyze_types/bigquery, [1, 2] is now typed ARRAY<INT64> where it used to be untyped.

Testing

• New end-to-end cases:
  • googlesql variants of create_view, ddl_alter_table_add_column, ddl_alter_table_alter_type, ddl_alter_table_drop_column and ddl_alter_table_rename.
  • ddl_spanner, a realistic schema with change streams, roles, grants, identity columns and vector indexes.
  • analyze_arrays, array_unnest, analyze_functions and func_nullability.
• Every new case's schema was checked against Spanner Omni. The new analyze_arrays and analyze_functions cases pass goldeneye's check against live Spanner. Nullability, string lengths and UNNEST lineage, which Spanner's wire types can't report, are pinned in the generate cases.
• go test ./internal/endtoend -run TestReplay/base passes in full. SQLC_TEST_CORE=1 … -run TestReplay/core has no new failures against main's baseline. The internal/core, internal/compiler, internal/engine and internal/cmd tests pass.
• goldeneye's spanner.TestDialect passes against Omni.
• The examples/authors/googlesql example passes against Spanner Omni, and its generated code is unchanged.
• goldeneye's analyze_types/googlesql still fails, as it does on main. sqlc names unattributed parameters after their @name, and goldeneye's Spanner answer leaves them blank.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L8QoGi6VRtADkb4jL3BFHP