chore: regenerate the API client from the current Client API spec

[quinn-galileo]

Why

The checked-in API client (src/splunk_ao/resources/, generated from openapi.yaml) has drifted from the server. Current servers return enum values the client doesn't know, and the generated parser raises on them. Against staging, Projects().get(id=...) fails with:

ValueError: 'configure_rate_limit' is not a valid ControlResourceAction

GET /projects/{id} now includes configure_rate_limit in permissions, which the generated ProjectAction enum lacks. Lookup by name still works, but every project lookup by id is broken on current servers. That blocks the project-scoped evaluator PRs stacked on this one.

Ticket: SAO-9342

What

The client is regenerated with the repo's documented procedure:

./scripts/import-openapi-yaml.sh https://api.galileo.ai/client
./scripts/auto-generate-api-client.sh

Spec source: the production Client API spec, https://api.galileo.ai/client/openapi.json. Nothing under resources/ is hand-edited.

Spec diff against the previous openapi.yaml:

• Paths: +6, −3.
• Schemas, counted by schema title: +33, −42, 94 changed. Counting raw component keys gives slightly different numbers, because the old CoreScorerName/ScorerName pair is now one schema.
• Enums: 18 changed.

Notable enum changes:

• ProjectAction: +configure_rate_limit, read_all_logs, read_filtered_logs; −update.
• CollaboratorRole: +custom.
• ScorerAction: +create_global, create_shared.
• UserRole: +capability_based. UserAction: +change_role_to_capability_based.
• NodeType and StepType: +http.
• OrganizationAction: +read_ai_usage_organization, read_unredacted_data.
• IntegrationProvider: +typesafe. The spec drops vegas_gateway; the import script keeps it (see below).
• ProjectType: −llm_monitor, −prompt_evaluation.
• TaskType: +19. The spec drops 7, 9 and 12; the import script keeps them (see below).
• ScorerName: now holds the public metric names, not the internal underscore-prefixed ones.

Import script. The spec now has a single ScorerName schema, which replaces the old CoreScorerName/ScorerName pair. The old CoreScorerName retitle patch no longer applied and failed the import. The duplicate-value dedupe now targets ScorerName, behind a guard that fails with a clear message if ScorerName, IntegrationProvider, LLMIntegration or TaskType is renamed or removed. Without the guard, a += patch on a missing schema would create a stub and silently drop the restored values.

• The spec dropped vegas_gateway from IntegrationProvider and LLMIntegration, and task types 7, 9 and 12 from TaskType. Older servers still return them, and the strict parsers would then fail: Integration.list(all=True) on any server still offering vegas_gateway, and Experiments().list() for a whole page containing one legacy experiment. Following the existing StepType patch, the script adds the values back (+= [...] |= unique).
• This also unblocks the scheduled "Update API Client" workflow, whose runs fail on the old CoreScorerName patch. Its stale PR Update API Client #205 will need closing or resetting before that workflow's next run after this merges.

Hand-written adaptations forced by the regeneration (all small):

• ScorerConfig was renamed RuntimeScorerConfig on the server. It keeps the old fields, adds new ones with defaults, and makes scorer_type optional. The references are updated in experiment.py, experiments.py, runs.py, scorers.py and utils/metrics.py, plus two tests. Before this change, import splunk_ao failed.
• ProjectCreate.create_example_templates was removed. Projects.create no longer passes it; passing it raised TypeError.
• The test that checks deletion of a non-gen_ai project now uses ProjectType.PROTECT, because LLM_MONITOR no longer exists.

The repo's mypy runs with follow_imports = "skip", so it can't see removed generated symbols, and neither break above was a type error. Both were found by importing every module and by resolving each hand-written import against the new tree.

Unchanged. The scope fields and endpoint that the stacked PRs use are generated with the same names and types:

• CreateScorerRequest.is_global / project_ids
• ScorerResponse.is_global / scope_projects (ScorerScopeProjectRef(id, name))
• UpdateScorerScopeRequest(is_global, project_ids)
• set_scorer_scope_scorers_scorer_id_scope_put

Breaking changes for SDK users

These matter only for code that imports generated symbols from splunk_ao.resources directly. The full list is in CHANGELOG.md under [Unreleased] → ### Changed.

• ScorerConfig → RuntimeScorerConfig. Its field order changed, so pass fields by keyword.
• Removed endpoints:
  • /projects/{project_id}/experiments/metrics
  • /projects/{project_id}/experiments/{experiment_id}/metric_settings
  • /integrations/vegas_gateway
• Removed models: the Customized*GPTScorer and *Template scorer families, FineTunedScorer, RegisteredScorer, ScorerType, ScorersConfiguration, and others.
• Removed fields: BaseScorerVersionResponse.input_type (returned by create_custom_llm_evaluator() and the other evaluator create functions); input_type, output_type and scoreable_node_types on BaseScorerVersionDB; BaseScorer.sub_scorers; ProjectCreate.create_example_templates; scorer_type / critique on the Metric* status models; and eight fields on CreateJobRequest / CreateJobResponse.
• Removed enum members: ProjectAction.UPDATE, ProjectType.LLM_MONITOR, ProjectType.PROMPT_EVALUATION. CoreScorerName is renamed to ScorerName, replacing the old underscore-valued ScorerName.

Where to start reading

1. scripts/import-openapi-yaml.sh: the one patch change.
2. The hand-written commit, fix: adapt hand-written code to the regenerated client, and the tests.
3. CHANGELOG.md.
4. openapi.yaml and src/splunk_ao/resources/ are mechanical generator output. The two chore: regenerate the API client … commits contain only those files.

import splunk_ao fails at the generated-only commit and works again after the next commit, so review the PR as a whole.

How it was tested

• New tests in tests/test_projects.py:

  • test_get_project_by_id_parses_configure_rate_limit_permission serves a project with a configure_rate_limit permission through respx and runs it through the real generated parser. It fails on the old client with the staging error above.
  • test_create_project_posts_body_without_example_templates checks the exact ProjectCreate body that Projects.create sends.

• New test in tests/test_experiments.py: test_list_experiments_parses_legacy_task_type serves an experiment with task_type: 7 through respx and checks that it parses.

• Targeted tests: test_projects, test_project, test_evaluators, test_evaluator, test_experiments, test_experiment, test_runs, test_scorers, utils/test_metrics and test_agent_streams_evaluators and test_integration all pass.

• poetry run invoke type-check passes. ruff check passes on the changed hand-written files.

• Every non-generated splunk_ao module imports.

• Read-only staging check, run from a script outside the repo:

  	Projects().get(name=...)	Projects().get(id=...)
  before (main client)	OK	ValueError: 'configure_rate_limit' is not a valid ControlResourceAction
  after (this PR)	OK	OK, 25 permissions parsed

Related PRs

Repo	PR	Purpose	Status
splunk/splunk-ao-python	this PR	Regenerate the API client (base of the stack)	draft
splunk/splunk-ao-python	#274	Scope evaluators to the current project on create; merges this PR in	in progress
splunk/splunk-ao-python	#275	Clear RBAC errors; merges this PR in	in progress

🤖 Generated with Claude Code

[github-actions]

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

[quinn-galileo]

🤖 This review was generated by the Astra agent (claude-opus-5-5). It may contain mistakes.

Verdict: approve — The hand-written adaptations are correct and complete. Every hand-written import resolves against the regenerated tree, and the regression tests exercise the real parser.

General Comments

• 🟡 minor (question): The linked ticket contradicts this PR's premise. SAO-9342 says: "API client: already done. ... No regeneration or api change needed." This PR regenerates the client and adds a Breaking entry under [Unreleased] (CHANGELOG.md:22). The staging configure_rate_limit failure justifies the change. Still, please confirm two points with the ticket owner:
• Should SAO-9342 be updated, or should this prerequisite get its own ticket? The description also says the link was not verified. The ticket exists and is under epic SAO-6584.
• Is a breaking change to splunk_ao.resources acceptable for the next release after 0.5.0 (pyproject.toml:3)? The stacked evaluator PRs will ship with it.

Follow-ups

Suggested follow-up work that could be tracked as Jira tickets:

• pyproject.toml:337-337: mypy runs with follow_imports = "skip", so it cannot detect a hand-written import of a generated symbol that a regeneration removed. This PR hit that twice: ScorerConfig and create_example_templates. Add a cheap guard test that imports every non-generated splunk_ao module and resolves each from splunk_ao.resources... import X. That catches the next regeneration break in CI rather than by hand. I ran an equivalent AST-based check on this branch, and it found no unresolved symbols.
• src/splunk_ao/evaluators.py:49-72: The regenerated DecisionScorerDB.answer_options now advertises two new server shapes:
• list[DecisionChoice] ({label, description}) for categorical.
• DecisionBooleanCriteria ({true, false}) for boolean_probability.
  normalize_decision_answer_options rejects both. A Mapping raises at line 49, and a choice dict without level raises at line 62. DecisionEvaluator._read_answer_options (src/splunk_ao/evaluator.py:1053-1061) then logs a warning and sets answer_options to None. Reads stay safe, but evaluators with described choices or boolean criteria lose that data. The create path also cannot send them. Add support for both shapes.

[quinn-galileo]

🤖 This review was generated by the Astra agent (claude-opus-5-5). It may contain mistakes.

Verdict: approve — The new enum guard fails on a spec that lacks the patched schemas and passes on the current spec. The corrected ground_truth note in CHANGELOG matches what RuntimeScorerConfig.from_dict does.