# PASS1 proof fixture matrix — Azure build-validation PR watches

Inherited capture specification refreshed to the final SHAs. Execution results are in index.md and runtime-transcript.md; this document retains the original row definitions, not a new sufficiency verdict.

## Identity and scope

- PR: [#18759 on GitCafe](https://git.cafe/pingdotgg/t3code/pulls/18759), [#18759 on GitHub](https://github.com/pingdotgg/t3code/pull/18759).
- Read-only audited checkout: `/home/ubuntu/wt/pr-18759`, branch `takeover/pr-18759`.
- AFTER: [e13265440a](https://github.com/pingdotgg/t3code/commit/e13265440a44b51116f9c3427b50c3d0e2c3ca1e), exact `e13265440a44b51116f9c3427b50c3d0e2c3ca1e`.
- BEFORE / origin/main: [3cc31ed27c](https://github.com/pingdotgg/t3code/commit/3cc31ed27c8956b36cee35a50509fb7c6869592b), exact `3cc31ed27c8956b36cee35a50509fb7c6869592b`.
- Original contributor commit: [da81b1d76d](https://github.com/AugusDogus/t3code/commit/da81b1d76d071cd52150c590d55ef4ae945b2054), exact `da81b1d76d071cd52150c590d55ef4ae945b2054`, branch `fix/azure-pr-watch-checks-18728`. This is provenance, not the BEFORE baseline.
- Origin is GitCafe; no forge CLI state was needed for the local proof audit. Working tree was clean at inspection.
- Authorized evidence: focused tests PLUS BEFORE/AFTER runtime transcript of the real PR checks path, using an executable stub `az` on PATH returning recorded Azure CLI JSON. No web server, browser, live watch, comments, pushes, or application database required.
- Required limitation label: **live Azure DevOps not exercised**. Missing live Azure credentials is not a blocker for this authorized evidence.
- Public evidence, harness scripts, and JSON fixtures belong outside the worktree and must be uploaded with `ghdrop upload -m`; never commit evidence scripts. Only sanitized, public-safe recordings may be uploaded. If a fixture is constructed rather than recorded, explicitly label it synthetic; do not call it a recording.

## Full diff inventory and affected callers

The five feature paths are under `packages/source-control-azure-devops/src/server/`; final CLI uses the named Effect.fn span. Unrelated main-merge changes are outside these proof rows:

| Changed path | Changed behavior / branches | Proof coverage |
| --- | --- | --- |
| `azureDevOpsPullRequestJson.ts` | New strict policy-page schema and `decodeBuildPolicyChecksJson`; build type filter, enabled/deleted/notApplicable filter, case normalization, status mapping, optional naming settings, blocking flag, raw page count, stable policy ID | D1–D4, R1–R3 |
| `AzureDevOpsPullRequestCli.ts` | New `listBuildChecks`; CLI argv, offset pagination by 100, max 1000 raw evaluations; decode/truncation/cap errors; no partial successful result | C1–C4, R1–R4 |
| `AzureDevOpsPullRequestProvider.ts` | `getChangeRequest` now calls checks after PR show and before iterations; returned detail includes checks; new optional `getChangeRequestPreview` reads only PR show; both map CLI failures through the existing provider error mapper | P1–P3, all runtime fixtures |
| `AzureDevOpsPullRequestCli.test.ts` | Updated old latest-head fixture for extra policy call; new mixed-policy second-page and unreadable-second-page cases | T1, C1, C3, P2 |
| `AzureDevOpsPullRequestProvider.test.ts` | Two existing detail mocks now supply empty build checks; latest-iteration and missing/error iteration behavior retained; new preview regression test returns usable preview while identical policy failure rejects full detail | T1, P2 |

Production call chain: `driver.ts` supplies real `AzureDevOpsCli` and PR CLI layers → Azure provider `getChangeRequest` → `PullRequestService.detailUncached` copies `changeRequest.checks` into wire detail → `PullRequestWatchReactor.readGroup` reads detail with `allowStale:false` → `evaluatePullRequestWatch` → `pullRequestWatchMessage` → reactor `record` / `woke` on nonempty changes. Real CLI execution is `AzureDevOpsCli.execute` → `SourceControlHost.process.run(command:"az")`.

Other affected callers (P3, traced at this head): `apps/web/src/components/ChatMarkdown.tsx:1274` renders `PullRequestLinkPreview`, whose `pullRequest/PullRequestLinkPreview.tsx:60` reads `pullRequestEnvironment.preview`; shared client-runtime `state/pullRequests.ts:203–208` preview query sends `WS_METHODS.pullRequestsPreview`; `apps/server/src/ws.ts:2532–2533` delegates to service preview. `PullRequestService.ts:533–535` preserves the optional provider reader, and `previewUncached` at 1789–1807 selects `getChangeRequestPreview ?? getChangeRequest`. Azure now supplies the former (`AzureDevOpsPullRequestProvider.ts:322–336`): PR show only, seven preview fields, no checks or iteration calls. Policy denial therefore cannot affect an otherwise readable Azure chat-link preview. A PR-show failure still maps to a preview error; it is not swallowed. Full detail at provider lines 338 onward still reads policies before iterations and fails closed. Azure summary uses separate `getChangeRequestSummary`, and list uses `listPullRequests`; neither gains policy reads. Azure still has no `getChangeRequestChecks` or `getChangeRequestWatchFingerprint`; service endpoints (2995 and 3023) return null for unsupported readers. Do not drive the separate checks endpoint and conclude the change is missing. No public contract or capability was added.

Refresh delta from prior PASS1 head [a4cdee52fa](https://github.com/pingdotgg/t3code/commit/a4cdee52fa5797298008a313ca974794e8cb5e29): 2 files, 52 insertions, 0 deletions (15 provider lines and 37 provider-test lines). This addresses review B's preview policy-read regression. It adds an existing optional provider method, without weakening full detail. Prior PASS1 remains a capture specification, not executed proof. This refresh also runs no tests or runtime captures.

## Fixture seed contract

Use a sanitized fixture directory, e.g. `/tmp/to-18759-proof/fixtures`, with manifest, source/provenance, SHA-256 for each JSON, and a separate executable stub directory. Prefix PATH only in the capture process. Stub must route by argv (not an accidental shared response queue): `repos pr show --id 42`, `repos pr policy list --id 42 --top 100 --skip N`, `devops invoke` resource `pullRequestIterations`, and iteration changes resource `pullRequestIterationChanges`. Log every invocation, cwd, scenario, sequence and selected JSON; reject unexpected commands. BEFORE must be allowed to make no policy call. No genuine `az` fallback or network access.

Common PR: active PR 42, source `refs/heads/feat/page`, target `refs/heads/main`, organization acme, project platform, repository web; valid PR-show fields from existing `pullRequestRow` fixture. Use `mergeStatus:succeeded` to prevent unrelated conflict news. Provide iteration 1 with head A (`aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`) and base B, changes `{ "changeEntries": [] }`. For head movement provide iteration 2/head C while PR-show lastMergeSourceCommit remains A. If needed, return a sanitized bare account username for `getViewer`, and empty threads; keep remarks empty so only checks cause changes.

Canonical build row: `status` string, configuration `{id:12,isEnabled:true,isBlocking:true,type:{id:"0609b952-1397-4640-95ec-e00a01b2c241"},settings:{displayName:"Tests"}}`. Extra fields such as policy `context.buildId`, evaluation IDs, build-result metadata may remain in recordings; current implementation ignores them. Replacement reads **policy evaluations**, not build-history or PR-status APIs. Include the corresponding recorded build result if available to explain the queued/running/failed/succeeded sequence, but do not invent `az pipelines build` calls in the harness: expect none. Expired/requeued policy state takes precedence over any old successful build result.

Seed watch state explicitly from `ThreadPullRequestWatch` in `pullRequestWatch.test.ts`: startedAt and remarksThrough fixed ISO timestamp, null headSha, failedChecks/passedChecks/remarkIds empty, passed/conflicting false, wakes 0. Feed each returned `report.next` into the next evaluation. Use provider capabilities unchanged (Azure `checksAlwaysReport` is absent), actual returned head/mergeability, and no remarks. Supply fixed logical times as function input; never sleep to simulate time.

## Matrix — affected branches, reduced combinations

Legend: **runtime minimum** means a BEFORE/AFTER transcript is required. **focused** means a deterministic focused test/assertion is enough; collapse these variants into table-driven assertions or the runtime fixture where convenient. Every row is server-only. An existing test listing is coverage found in source, not a claim it passed.

| ID / capture level | Seed / interaction | Expected BEFORE | Expected AFTER / assertions |
| --- | --- | --- | --- |
| D1 focused; folded into R1 | Approved, rejected, broken, queued, running, unknown status on enabled build rows; include uppercase `APPROVED`, `REJECTED`, build UUID, `NOTAPPLICABLE` | Provider checks always empty | approved→success; rejected/broken→failure; queued/running/unknown→pending; case-insensitive comparisons; notApplicable omitted. Unknown cannot pass. Decoder direct assertions cover broken/unknown not present in new tests. |
| D2 focused; folded into R2 | Mixed non-build policy, disabled build, deleted true build, notApplicable build, plus enabled active build. Vary isDeleted absent/null/false | Empty checks | Only enabled, nondeleted, applicable build rows emitted. rawCount includes every decoded row, including filtered policies. Malformed filtered rows still fail strict schema rather than being silently skipped. |
| D3 focused; folded into R2 | Two policies named Tests IDs 12,13; same policy ID with changed build/evaluation IDs; settings omitted/null/{}, displayName omitted/null/blank/padded string; isBlocking true/false | Empty checks | Names Tests (#12), Tests (#13); ID survives rerun; trimmed labels; missing/null/blank name→Build validation (#id); required exactly boolean isBlocking; description/url null. Collapse nullable name variants into one table. Display-name change can change identity: do not claim rename suppression beyond current name+policy ID behavior. |
| D4 focused; folded into R3 | Invalid JSON, top-level object, null row, missing status/config/type/id/isEnabled/isBlocking, wrong scalar type or noninteger ID; valid unrecognized extra fields | Policy JSON never requested | Invalid page fails as a whole; valid extra fields accepted. Include malformed non-build row to prove strict-before-filter branch. No partial success. |
| C1 **runtime minimum R2** | 100 nonbuild policies page skip=0, then 3 enabled build rows (rejected required 12, approved required 13, running optional 14), disabled and notApplicable extras | No policy CLI call, empty checks | Calls skip=0 then 100 even though first page emitted zero checks; 3 checks with correct fields. CLI argv includes detect true, id 42, top 100, only-show-errors, output json, correct cwd. Existing new mixed-page test covers core case. |
| C2 focused | Empty first page; 99 raw rows; exactly 100 then empty page; 999 rows (9 full +99) | Empty checks | Empty/short terminates, exact boundary requests next page; 999 succeeds. Ten full pages (1000 raw rows) fail read limit after skip 900, do not return passing subset or request skip 1000. Full pages may all filter out. No max-cap test currently added. |
| C3 **runtime minimum R3** | 100 approved rows then second page `{}`; repeat with invalid JSON/malformed row; overflow output to trigger stdoutTruncated even if collected prefix is valid JSON | Policy errors invisible; detail still succeeds with checks [] | Whole getChangeRequest fails, provider reason failed; never evaluate/emit passed from page 1. Existing new test covers `{}` and explicit truncated flag, not real subprocess overflow. Test actual overflow via oversized JSON/whitespace exceeding process collector limit, or clearly label flag injection as focused-only. |
| C4 focused; runtime can fold into R3 | Policy command exits nonzero: authentication, rate limit, not-found, generic; missing az executable | No extra command introduced | Existing mapper reasons unauthenticated/rate-limited/not-found/failed/missing-tool retained. At least generic command failure must be demonstrated or covered by focused assertion; no partial checks. Rate limiting pauses reactor rather than consuming read-failure count. Avoid duplicating every existing generic Azure error test if unchanged mapping is already covered. |
| P1 **runtime minimum R1/R2** | Valid PR→policy→iterations→changes through real layers | PR→iterations→changes; checks [] | Policy runs between show and iterations; actual detail checks feed unchanged watch evaluator and message formatter. Required checks form pass gate; advisory pending cannot block required pass; advisory failure is still reported. |
| P2 focused; **runtime minimum R4** | Latest iteration C, PR trial head A; iterations missing/error; location null; successful checks in each | Existing head/changedFiles fallback behavior | C still wins trial A; no iterations/error retain fallback and checks; location null skips iteration calls but still reads policies. Original focused latest-head test and two provider mock changes cover parts. Policy failure occurs before iteration fallback and must not be swallowed. |
| P3 caller trace + **runtime folded into R3** | Full detail versus optional preview reader with policy command denial; separate summary/list/checks/fingerprint endpoints | No policy read anywhere; preview service falls back to full detail, which still succeeds with checks [] | Detail performs policy call and fails; preview calls only PR show and returns seven usable fields. No iterations or policy calls for AFTER preview. PR-show errors still propagate. Summary/list unchanged; checks/fingerprint remain null. Trace above establishes service selection; R3 exercises production provider/CLI boundary without UI or service harness expansion. |
| W1 **runtime minimum R1** | Same head: running→rejected→same rejected→queued→approved→same approved; sequential logical times | checks [] and no checks-failed/checks-passed at every step | pending/no wake; one failure; duplicate suppressed; rerun pending clears failed state/no pass; one required pass; duplicate pass suppressed. Record exact checks, changes, next watch and formatted wake text. |
| W2 **runtime minimum R2** | Required 12 rejected→approved while required 13 approved and optional 14 running; optional later approves; two labels equal | No check wakes | Initial failure; required pass count 2 even while optional pending; owed optional completion produces final pass count 3 required false; distinct IDs avoid collision. Repeated unchanged read yields no extra wake. |
| W3 focused; fold into R3 | failure already reported→unreadable policy page→same failure→running→failure | No check results | On read error no evaluator call, previous watch unchanged; same failure still suppressed after recovery. A genuine pending rerun permits a fresh failure later. Reactor eventually gives up after its existing consecutive-read-failure limit; do not demand this unchanged limit as another runtime recording. |
| W4 **runtime minimum R4** | Head A passed; next actual iteration C pending with stale trial A; C passes | Head still updates, no check pass/fail | C resets remembered pass/fail state, pending blocks pass; then one new pass at C. Provide optional empty/all-filtered policy observation at logical +16min using actual Azure capabilities | 
| W5 focused / folded into R4 | Empty/all-filtered checks, Azure checksAlwaysReport absent; logical time +16min | No check pass/fail or no-checks notice | Same; do not invent no-checks timeout wake for Azure. An empty list cannot establish pass and preserves previously reported check state on the same head. Capabilities untouched. |

## Minimum capture package for next agent

1. **T1 focused tests**: capture commands, exit codes, commit and full focused output. Run only `AzureDevOpsPullRequestCli.test.ts`, `AzureDevOpsPullRequestProvider.test.ts`, `azureDevOpsPullRequestJson.test.ts`, and `apps/server/src/orchestration-v2/pullRequestWatch.test.ts` using `vp test run <paths>`. New CLI tests should fail against BEFORE because checks are empty / unreadable policies do not fail detail; record the actual failures, not presumed ones. The unchanged watch/JSON suites are regression context. Do not run repo-wide checks. Branch cases not represented in committed tests may be checked by the external replay script; label those assertions separately rather than claiming repository coverage.
2. **R1 lifecycle**: one BEFORE and one AFTER run of W1 using identical recorded policy inputs and logical times; log real subprocess az argv, provider checks, watch changes/state and `pullRequestWatchMessage` output on each event. Include correlated build-result recording if supplied; evaluations are authoritative. This is the central bug reproduction and fix demonstration.
3. **R2 paging + mixed gate**: paired run of C1/W2, including disabled/deleted/notApplicable/non-build data and fallback label where feasible. Show raw page sizes and skip values, duplicate-name policy IDs, then required pass and optional settlement.
4. **R3 fail-closed**: paired run with passing page 1 and unreadable page 2. AFTER failure must preserve previous watch state and produce no false passing message. Add focused assertions for truncation and cap plus malformed row, and log generic command failure if feasible. BEFORE remains readable but has no checks; do not present it as having a false pass it never emitted. **Fold P3 preview isolation into this same R3 section:** configure the stub policy command to exit nonzero with synthetic `policy read denied` stderr and PR show to succeed. In fresh uncached invocations, request preview then full detail. AFTER must return exactly number/title/url/author/state/isDraft/createdAt from preview, with only PR-show argv logged, then fail full detail at the policy command with no watch evaluation, no formatted wake and unchanged prior watch. BEFORE has no optional preview reader: use the same service selection `(provider.getChangeRequestPreview ?? provider.getChangeRequest)` and project the seven preview fields; fallback reads show/iterations/changes but never policies, and full detail also succeeds with checks []. Log each call boundary separately so a denied command configured but never invoked is distinguishable from a real failure. This is one additional phase of R3, not R5. Label fixtures synthetic unless sourced from an actual sanitized recording. The new committed provider test is focused mocked coverage, not this real subprocess proof.
5. **R4 push and no checks**: paired run for P2/W4/W5. Show actual latest iteration C beating stale PR trial A, reset then pass, and no fabricated no-checks notice after logical +16 minutes.

Four compact runtime sections may share a harness and fixture files. Table-driven D/C/P boundary assertions may share the AFTER runtime harness. Do not create a Cartesian product of all statuses × all nullable labels × pagination × head changes. R1 isolates time/state, R2 pagination/gating, R3 errors plus preview isolation, R4 head and empty data. Existing unchanged watch tests cover comment watermarks, conflict handling, wake budgets and other host behavior; those do not need Azure runtime duplicates.

## Execution boundary and evidence honesty

Use production `AzureDevOpsPullRequestProvider.make`, `AzureDevOpsPullRequestCli.layer`, and `AzureDevOpsCli.layer`; give them a host that launches the stub executable for real. Existing `TestSourceControlHost.layerWithRealGit` launches real subprocesses and can supply this boundary without a running T3 server. Keep its default process runner; mocking `AzureDevOpsCli.execute` would be a focused test, not the requested PATH-stub runtime. Run a fresh process for each revision so imports/cache cannot mix BEFORE and AFTER. Resolve imports/dependencies from the owning package or server, not repo root. BEFORE source extraction must preserve module resolution and must not modify the audited worktree; coordinate any baseline export outside it with the dispatcher.

For the watch portion, pass real returned detail fields plus unchanged provider capabilities to `evaluatePullRequestWatch`, carry `next`, and call real `pullRequestWatchMessage` when changes exist. Clearly label this **runtime provider-to-watch decision/message transcript**: it proves wake decisions and formatted text, not a durable thread enqueue or a provider agent starting. Existing reactor code trace establishes the `changes → record → woke` connection; include a focused existing reactor test if claiming persistence. No live watch or server is authorized here.

Each transcript needs: exact source SHA; sanitized fixture hashes/provenance; runtime and command; stub path and proof it handled calls; event index/logical timestamp; selected policy/build-result state; actual provider checks or actual typed error; watch state before/after; change kinds; formatted message (or explicit none); assertion outcomes and exit code. Record actual time only as metadata. Sequence transitions are explicit fixture switches between completed reads, not sleeps or polling. Reset scenario state between revisions. Never hide errors with a successful script exit or replace real output with expected text.

Upload the matrix, replay script, sanitized fixture bundle/manifest, focused test logs and BEFORE/AFTER transcript(s) using `ghdrop upload -m`. Keep all originals outside the checkout. Report every returned URL and local path. Capture agent must retain exact SHAs and the label **live Azure DevOps not exercised**. Do not commit any evidence or scripts, modify code, push, comment, watch, start servers or open browsers under this brief.

## Render/surface accounting

No frontend file changed. Preview isolation changes server behavior consumed by the existing chat-link preview; its render path is traced in P3 and requires no new visual capture for this server-only brief. Web (local/public hosted), desktop's wrapped web and Electron shell, mobile React Native, marketing, shared client-runtime, Settings, chat, command palette and keybinding render paths are unchanged. Existing PR detail/check displays and notifications may consume the newly populated server fields; this is a data correction through existing contracts, not new rendering. No screenshot, video, UI fixture, mobile fixture or browser pass is required for this server-only scope. Client connection modes use the same server detail contract; no transport/origin code changed. Coding-agent providers all receive the same watch message through the existing reactor; none of their adapters changed. Other source-control providers, actions, summaries, review activity and scheduled-task transport are unchanged. Do not extrapolate new surface requirements from them.

Friction: **tool** — two bounded caller searches used nonexistent guessed source paths; corrected by searching actual package/component paths. No capture or upload blocker. No forge state was read or refreshed; this is the explicitly requested local exact-head proof-spec refresh. Live Azure credentials are unavailable by brief and are a declared scope limit, not friction or a blocker. This document requests capture; it does not certify unrun evidence or give a PASS2 verdict.