Skip to content

feat(validator): add a GitHub validator and select each queue's validator in the orchestrator profiles - #878

Open
roychying wants to merge 1 commit into
mainfrom
chenghan.ying/github-validator
Open

roychying wants to merge 1 commit into
mainfrom
chenghan.ying/github-validator

Conversation

@roychying

@roychying roychying commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Why?

SubmitQueue only rejects a GitHub pull request at validation for what the merger's mergeability check catches: closed, draft, conflicting, wrong stack shape or target. A pull request that is missing a required review or has a failing or still-running required check is accepted, batched and built. It then fails at land time, when GitHub refuses the merge, after holding up the queue.

Every queue in the example orchestrator also gets the always-passing fake validator, with no way to choose a real one per queue.

What?

submitqueue/extension/validator/github: a validator.Validator that asks GitHub whether each pull request could merge under the repository's own branch protection or rulesets. Nothing is restated in SubmitQueue config. For each github:// URI, bottom of the stack first, it reads the pull request in one GraphQL call and rejects at the first rule it breaks:

Rule Reason
URI names the configured host host
Pull request head is still the URI's commit stale
Review decision is not CHANGES_REQUESTED / REVIEW_REQUIRED not_approved
Every required check passed (success, neutral and skipped count as passing; a check name reported more than once must pass every time) check_failed / check_pending
Merge state is not BLOCKED (catches what the rules above can't see, e.g. a required check that hasn't reported yet) blocked
  • Pass when undecided: anything GitHub hasn't decided (UNKNOWN merge state, no review rule) passes. GitHub enforces the same rules again when it merges.
  • No duplicated checks: open, draft, conflict and stack shape stay with the merger's mergeability check. A missing pull request stays with the change provider.
  • Composes with other providers: URIs that aren't GitHub change IDs are skipped, so it combines with other change providers' validators via composite.
  • Credential and base URL come from the caller: they arrive on the injected *http.Client, the same contract as the GitHub change provider and merger.
  • Errors: a rejection is an error naming the pull request and rule. A failure to read GitHub is returned wrapped, keeping platform/http.StatusError in the chain for platform/errs/http. Neither is classified here, as for any extension.
  • Metrics: tagged with the queue. validate.finish counts only read failures as errors. validate.rejections is also tagged by reason.

Orchestrator wiring (service/submitqueue/orchestrator/server):

  • Profiles gain a per-queue validator: block, {type: fake|github} with tokenEnv, baseUrl, host and timeout. These default to GITHUB_TOKEN, https://api.github.com and github.com.
  • The default is fake, so existing queues and the built-in example setup are unchanged.
  • This replaces the hard-coded validatorFactory in main.go.
  • A github validator whose token is unset fails at startup.

Test Plan

bazel test //submitqueue/extension/validator/... //service/submitqueue/orchestrator/server:all
make lint check-gazelle check-mocks check-tidy

Issue

@roychying
roychying requested review from a team, behinddwalls and sbalabanov as code owners October 16, 2026 18:41
@roychying
roychying added this pull request to stack #880 October 16, 2026 18:43
@roychying
roychying force-pushed the chenghan.ying/github-validator branch from d569932 to d803dd4 Compare October 16, 2026 18:57
@roychying
roychying force-pushed the chenghan.ying/github-validator branch from d803dd4 to 80a785b Compare October 16, 2026 18:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant