Skip to content

runners: gh CLI (GH-A1–A22) + git transport (G1–G12) + conformance CI - #2

Open
tamnd wants to merge 1 commit into
mainfrom
feat/runner-gh-cli
Open

tamnd wants to merge 1 commit into
mainfrom
feat/runner-gh-cli

Conversation

@tamnd

@tamnd tamnd commented Jun 10, 2026

Copy link
Copy Markdown
Owner

What

Shell-based runners for the two most foundational client ecosystems, plus the full conformance CI workflow matrix.

gh CLI runner (runners/shell/gh_cli_test.sh)

Covers all 22 GH-A* gates:

Gate Command
GH-A1 gh auth status --hostname HOST
GH-A2 gh repo view --json name
GH-A3 gh repo list --json name
GH-A4 gh repo create (creates + deletes temp repo)
GH-A5 gh issue list --json number
GH-A6 gh issue create
GH-A7 gh issue view --json title
GH-A8 gh pr list --json number
GH-A9 gh pr create (skipped if no feature branch)
GH-A10 gh pr view --json number,mergeable,title
GH-A11 gh pr checkout (skipped outside git repo)
GH-A12 gh pr diff (checks for diff --git prefix)
GH-A13 skipped (preserves fixture PR)
GH-A14 skipped (needs separate PR)
GH-A15 gh pr checks
GH-A16 gh release create
GH-A17 gh release upload (4KB random binary)
GH-A18 gh api user
GH-A19 gh api --paginate
GH-A20 gh api graphql (viewer { login })
GH-A21 gh search repos
GH-A22 gh label list

Uses GH_ENTERPRISE_TOKEN + GH_HOST env vars so gh auth is non-interactive in CI.

git transport runner (runners/shell/git_test.sh)

Covers G1–G12:

  • G1/G2: HTTPS/SSH clone + git fsck
  • G3/G4: HTTPS/SSH push (new branch per run)
  • G5: force push (amend + push --force)
  • G6: git fetch origin refs/pull/N/head
  • G7: shallow clone (--depth 1); verifies exactly 1 commit
  • G10: protocol v2 via GIT_TRACE_PACKET
  • G11: LFS batch endpoint returns 501 + JSON body
  • G12: deploy key SSH clone

SSH and deploy-key gates skip when GITHOME_DEPLOY_KEY_PATH is not set.

Conformance CI (.github/workflows/conformance.yml)

  • workflow_dispatch with host, token, runner inputs
  • Daily cron at 02:00 UTC (uses repo vars/secrets)
  • One job per client ecosystem, each uploads a *.tap artifact
  • report job collects all artifacts, prints gate summary

Testing locally

export GITHOME_HOST=githome.example.com
export GITHOME_TOKEN=ghp_xxx
export GITHOME_OWNER=my-org
export GITHOME_REPO=test-repo

make run-gh    # GH-A1 through GH-A22
make run-git   # G1 through G12

runners/shell/gh_cli_test.sh (GH-A1 through GH-A22):
  - auth status, repo view/list/create
  - issue list/create/view
  - pr list/create/view/diff/checkout/checks
  - pr view --json with mergeable field
  - release create/upload
  - api /user (REST + GraphQL)
  - api --paginate
  - search repos, label list
  Skips destructive gates (pr merge, pr review --approve) to
  preserve the fixture PR across runs.

runners/shell/git_test.sh (G1 through G12):
  - HTTPS and SSH clone/push/force-push
  - PR ref fetch (refs/pull/N/head)
  - shallow clone
  - git protocol v2 (GIT_TRACE_PACKET check)
  - LFS stub (expects 501)
  - deploy key SSH
  Injects token into URL via git -c url.insteadOf to avoid
  storing credentials in git config.

.github/workflows/conformance.yml:
  - workflow_dispatch inputs: host, token, runner
  - daily schedule at 02:00 UTC
  - matrix jobs: git, gh-cli, octokit-js, python, go-clients,
    ruby, dotnet, java, terraform
  - aggregate report job collects all *.tap artifacts
@tamnd
tamnd force-pushed the feat/runner-gh-cli branch from a2942cf to 336939b Compare June 10, 2026 04:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant