# Proof audit, pass 1 — fixture matrix

Findings first: **no missing or uncapturable path in the matrix below.** This is a matrix-completeness verdict, not a runtime-proof or merge verdict. No new capture, test run, server, browser, edit to the worktree, push, or PR comment was made in this pass.

PR: [#25675](https://git.cafe/pingdotgg/t3code/pull/25675), branch `fix/issue-25636`, reviewed head [802efb97da](https://git.cafe/pingdotgg/t3code/commit/802efb97da70e935d2c55ff2be6caf1093ffdb32), requested merge-base [96fc7b9551](https://git.cafe/pingdotgg/t3code/commit/96fc7b955196637605cbf3b415152ce4dd0e2294). Read-only worktree: `/home/ubuntu/wt/pr-25675`. GitCafe metadata initially confirmed that head and target `main`; later metadata retries had a transient connection failure, recorded below. The initial status had 39 successful checks, none pending/failing; CI is context, not proof of the matrix.

## Findings and proof constraints

1. **The changed execution path is stop-only.** The complete requested diff is 24 additions/1 removal in `EffectWorker.ts` and 88 additions in `FoundationPersistence.test.ts`. `boundExecution` runs after claim and the durable cancellation recheck, around each individual executor attempt, inside the existing cancellation race. Success, original failure, defect, retry/backoff, terminal failure, and cancellation continue through existing settlement branches. There is no client rendering path.
2. **The new test is narrower than the required runtime recording.** Its fresh/replayed variants use a real outbox with a synthetic interruptible never-returning executor and TestClock. They assert eventual later-start execution plus a failed stop containing `Timed out`. They do not directly establish 30 seconds per attempt, five-attempt accounting, backoff, or the stop status at the instant the later start executes. R1/R2 recordings must retain the ordered durable transitions and executor-entry milestones that establish these facts. A virtual one-hour test advance is not a runtime capture.
3. **A timeout is not a universal hard wall-clock bound.** Effect timeout interrupts its losing execution and waits for interruption/finalization. An uninterruptible region, hanging finalizer, or synchronously blocked JS thread can still prevent settlement. The matrix's hang fixtures must explicitly say *interruptible executor hang*. The original incident's underlying hang cause is not established by this PR. This is a disclosed scope limit, not evidence that the scoped fix fails.
4. **Direct MCP subtree walks are outside the wrapper.** MCP task cancellation dispatches `thread.stop` with `stopDelegatedTasks:false` and calls the recursive service itself. That direct call gains no timeout from this diff. Native-provider task/workflow stop paths also are not interchangeable with app-owned delegated-stop effects.
5. **Old evidence is input, not a current capture.** I downloaded the supplied [earlier index](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/5c43180015217c90/evidence-index.md) and relevant rebuild/driver scripts only after tracing this head. Independent `git diff --no-ext-diff --quiet` checks exited 0 for both changed files between [802f44312e](https://git.cafe/pingdotgg/t3code/commit/802f44312e952d65129ab31afb19a990757bb142) and this head, and for the whole orchestration-v2 directory between its older base [394929af76](https://git.cafe/pingdotgg/t3code/commit/394929af7687886084234ccb92b744f51ede271b) and the requested base. That corroborates the row design; it does not relabel historical recordings as recordings of the requested exact revisions. Pass 2 must bind its evidence to the requested before/after revisions, or explicitly assess a wider provenance bridge.

## Derived fixture matrix

“Recording” means a timestamped transcript of the action/time-dependent execution, durable outbox transitions, and relevant persisted events. “Ready” means the fixture is capturable; **nothing here claims a new capture exists**. The seven minimum recording rows each require before and after: **14 recordings**, with no UI screenshots. R0 is scope inventory; R6 is an optional deterministic race recording, with the unchanged branch covered by source inventory for the minimum set.

| Row / named fixture | Data and action needed | Expected before → after; distinct observable state | Diff affects row? | Kind / minimum set | How to seed; status |
| --- | --- | --- | --- | --- | --- |
| R0 — no wrapped stop / bypass inventory | Inspect both effect producers, stop eligibility, direct service call sites, and the executor union. Inputs: no app-owned child; native child; null child ID; deleted thread; `stopDelegatedTasks:false`; plain `run.interrupt` without `holdQueue:true`. | No eligible stop is enqueued, or the subtree service is invoked directly. Neither revision acquires this worker timeout on those calls. A held `run.interrupt` can enqueue a no-op stop even with no children; it belongs to R3, not this exclusion. | No | Source inventory; no minimum recording | Immutable source only. Optional app actions through `orchestration.dispatchCommand` can corroborate eligibility. **Complete inventory; no capture required.** |
| R1 — fresh interruptible stop never returns | Fresh pending stop (`attempt_count=0`), production daemon defaults, executor entry/interrupt logs, then a real `message.dispatch` creating later same-thread `provider-turn.start`. Inject only the stop executor's outcome as interruptible `Effect.never`. | Before: attempt 1 stays running beyond lease expiry; later start stays pending/0, no reply. After: claims 1–5 each time out after 30s, retry delays 100/200/400/800ms, then stop `failed`; only then later start is claimed and a real assistant reply/run/checkpoint settles. Roughly 151.5s plus overhead with cooperative cleanup. | Yes: core fix | **Recording; required** | Real isolated server creates project/thread/DB through authenticated RPC. With that owned server stopped, seed one pending stop plus an append-only audit trigger in the disposable DB; restart and send a real message. Clearly label the seeded stop and executor injection. Baseline observation ≥160s from claim. **Ready; not captured.** |
| R2 — process loss and replayed interruptible hang | Same as R1; claim attempt 1, optionally show an old provider start pending, kill only the recorded owned server PID, restart unchanged DB. Disable automatic restart continuation for the fixture; dispatch a new message after recovery. | Before: recovery requeues replay-safe stop, claim becomes attempt 2, hangs again. After: attempts 2–5 get fresh 30s bounds; remaining retry delays are 200/400/800ms, stop fails, new post-restart message gets a real reply. Pre-crash process-bound pending starts are cancelled by recovery; do not count them as the recovered start. Roughly 121.4s plus overhead after replay claim. | Yes: replay regression | **Recording; required** | Real isolated app/process crash and recovery, not merely a `replayed` flag. Preserve claim-before-exit, exit, startup reconciliation, replay claim and new-message boundaries. No DB rewriting between crash and restart. Baseline observation ≥130s after replay claim. **Ready; not captured.** |
| R3 — genuine delegated stop succeeds before deadline | One genuine app-owned child with non-null child thread ID and running provider turn; active parent; normal `thread.stop` with a distinctive reason; then a real new parent message. No executor injection. Zero children via held `run.interrupt`, duplicate child records, grandchildren and absent reason are equivalent service-input variants, not extra worker states. | Both: genuine recursive stop service returns before 30s, stop succeeds on attempt 1, no timeout/retry, child Stop dispatch is observable and next parent message completes. Child provider interrupt settlement can interleave across threads; service return does not promise all child effects have settled. | Wrapped, outcome preserved | **Recording; required** | Authenticated RPC: create project/thread, `message.dispatch`, `delegated_task.request` (omit optional workspace strategy for immediate start), await actual child-running event, `thread.stop`, then `message.dispatch`. A controllable provider terminal/HTTP gate can keep parent/child running. **Ready; not captured.** |
| R4 — timeout followed by successful retry with a fresh deadline | Seeded stop; attempt 1 interruptibly never returns; attempt 2 waits 25s then invokes genuine stop service; real later message queued behind it. Keep worker policy unchanged. | Before: first attempt still hangs past 65s. After: attempt 1 timeout, 100ms retry, attempt 2 succeeds after its own 25s execution (total about 55.1s); no third attempt. Later start remains blocked during the retry backoff and running second attempt, then real reply completes. This distinguishes per-attempt timeout from a deadline shared across attempts. | Yes | **Recording; required** | Same real app/bootstrap/outbox seed as R1, explicitly labelled executor outcome injection. Reason omitted or present does not change deadline. **Ready; not captured.** |
| R5 — original executor failure before deadline | Stop attempt 1 returns a uniquely identifiable `OrchestrationEffectExecutionError` after a short 5s gate; attempt 2 returns genuine success; queue a real later message. | Both: warning and pending retry retain original cause, not `Timed out`; 100ms backoff; second claim clears `last_error`, then succeeds; next message completes. Original-error exhaustion/defect variants share the same unchanged retry/fail branches; see branch inventory below. | Wrapped, original failure preserved | **Recording; required** | Real server with seeded stop and labelled executor error injection. A naturally failing service dependency is not a stable app setting; use a controlled injection, never change timeout/backoff. **Ready; not captured.** |
| R6 — durable cancellation before executor entry | Claimed stop gets durably cancelled between claim registration and the worker's reread. Executor count/entry milestone must remain zero; optional pending same-thread successor. | Both: durable recheck clears cancellation registration and returns; no executor, timer, timeout error or retry; cancelled row remains cancelled. Already cancelled pending rows are never claimed at all. | No: short-circuit precedes wrapper | **Recording if exercised; optional**, source inventory sufficient for minimum | No stable UI/RPC action can deterministically hit this race. Label as production-worker harness; use a barrier around `claimNext`/reread and the real outbox cancellation service. Existing generic claim-cancellation regression identifies the unchanged branch but is not a stop runtime capture. **Ready as labelled harness; inventory complete.** |
| R7 — durable cancellation of an actively running stop | Stop running on attempt 1 with interruptible never-returning executor; same-thread successor pending. Before 30s, call real outbox `cancelUnsettled` then `signalCancellations`; log executor interruption and `clearCancellation`. | Both: running → cancelled, executor interrupted, registration cleared; no timeout retry; successor can run. Keep daemon alive beyond original deadline (≥35s from initial claim) and verify row stays cancelled, attempt 1. Distinguish durable cancellation from killing the worker process. | Wrapped, cancellation semantics must be preserved | **Recording; required; labelled harness** | There is no normal command that promises durable cancellation of this stop type (current user stop cancellations target other effect types). Use production worker + real file-backed SQLite outbox + synthetic executor, on a DB migrated by the real app; app server stopped during harness ownership. Successor is a synthetic start; do not claim a provider response. **Ready; not captured.** |
| R8 — non-stop execution exceeds 30s | Real worker/outbox with `checkpoint.capture` synthetic executor held by a Deferred for 35s; later same-thread synthetic start pending. Capture first lease expiry, then release the gate. | Both: checkpoint remains running attempt 1 beyond 30s and lease expiry; successor pending/0; no timeout/retry/reclaim. Explicit release succeeds checkpoint and then successor. This proves the false branch of the stop-only type gate, not checkpoint implementation behavior. | No: identity branch | **Recording; required; labelled harness** | Same labelled worker/SQLite harness as R7. Use real worker defaults; executor outcome only is synthetic. All other non-stop types collapse to this identity branch; list below. **Ready; not captured.** |

## Complete caller and branch inventory

Source anchors are at the reviewed immutable head:

- [EffectWorker.ts](https://git.cafe/pingdotgg/t3code/blob/802efb97da70e935d2c55ff2be6caf1093ffdb32/apps/server/src/orchestration-v2/EffectWorker.ts#L542): stop adapter → `threads.stopDelegatedTasks`; timeout helper at line 568; shared `runOnce` at line 727; wrapper/cancellation race at line 771; settlement and retries at lines 784–832. `runOnce`, `runRecoveryOnce` (excludes restart continuations), and `drain` all call that shared implementation. The production daemon calls `runOnce`, default concurrency 4; tests can drive it with concurrency 2. There is no second executor call site inside this worker that omits the helper.
- [Orchestrator.ts](https://git.cafe/pingdotgg/t3code/blob/802efb97da70e935d2c55ff2be6caf1093ffdb32/apps/server/src/orchestration-v2/Orchestrator.ts#L558): exactly two effect-producer paths. `thread.stop` (line 10562) emits after successful interruption or fallback/idle handling if the thread is not deleted, delegation is not explicitly disabled, and an app-owned child thread exists. `run.interrupt` (line 11933) emits when `holdQueue:true`, after optional orphan repair; the plain interrupt path does not. Both use the same factory and optional reason payload, with deterministic effect ID and command/thread IDs. These caller states change whether and when the row exists, not the timeout behavior once claimed; R0/R3 inventory them and R1/R2/R4 cover its actual execution states.
- [ThreadManagementService.ts](https://git.cafe/pingdotgg/t3code/blob/802efb97da70e935d2c55ff2be6caf1093ffdb32/apps/server/src/orchestration-v2/ThreadManagementService.ts#L1165): recursively visits app-owned non-null child threads, deduplicates reused child IDs, dispatches child stops with recursion disabled, then visits descendants. It logs individual failures and returns the first after traversal. Empty/native/null-child inputs return successfully; any child count, duplicate IDs, depth, optional reason, or service failure feeds the same outer success/error/timeout classification. One genuine child is the minimum R3 fixture; a grandchild is useful optional service evidence, not another changed worker path.
- [OrchestratorMcpService.ts](https://git.cafe/pingdotgg/t3code/blob/802efb97da70e935d2c55ff2be6caf1093ffdb32/apps/server/src/mcp/OrchestratorMcpService.ts#L2371): task cancellation calls the service directly after child Stop, bypassing `boundExecution` (R0). The recursive service's own recursive call is also not separately wrapped; one worker attempt's timeout covers the entire service traversal it starts.
- [EffectOutbox.ts](https://git.cafe/pingdotgg/t3code/blob/802efb97da70e935d2c55ff2be6caf1093ffdb32/apps/server/src/orchestration-v2/EffectOutbox.ts#L317): pending/running non-title rows serialize per thread; an earlier pending row blocks even during retry backoff. `thread-title.generate` is a separate lane; other threads are independent until worker capacity is exhausted. The claim predicate requires pending status and does not reclaim running leases in-process. Settled stop states (`failed`, `succeeded`, `cancelled`) no longer block later work. R1/R4/R7/R8 must show same-thread ordering, not rely solely on final snapshots. Optional title/other-thread sentinels can share R1 without a new fixture.
- [FoundationPersistence.test.ts](https://git.cafe/pingdotgg/t3code/blob/802efb97da70e935d2c55ff2be6caf1093ffdb32/apps/server/src/orchestration-v2/FoundationPersistence.test.ts#L2597): exactly two added variants (`replayed:false/true`), corresponding to R1/R2. Replay setup claims then reconciles a stop; the runtime R2 strengthens that by using an actual process exit/restart. No production provider is exercised by these two tests.

### Effect types and data equivalence

`delegated-tasks.stop` is the sole true branch. Its request contains only the discriminator and optional string `reason`; IDs, reason length/absence, children counts and provider choice do not select a different worker timeout branch. The stop service can return successfully with zero eligible children, successfully after one/many/deep children, fail with its own error, die, or hang. These collapse to R3, R5, or R1/R4 according to outcome, rather than requiring a Cartesian product of service internals.

The 14 false-branch types are `provider-runtime.continue`, `provider-session.detach`, `provider-turn.start`, `provider-turn.interrupt`, `provider-turn.steer`, `provider-turn.restart`, `runtime-request.respond`, `provider-thread.rollback`, `checkpoint.capture`, `scheduled-tasks.cleanup`, `terminal.cleanup`, `preview.cleanup`, `attachment.cleanup`, and `thread-title.generate`. Their individual payload shapes affect their unchanged adapters, not this identity branch. R8 is the minimum >30s identity-branch control; the real `provider-turn.start` successors in R1–R5 also exercise another false-branch type.

Web/local-hosted web, desktop, mobile, Codex/Claude/Cursor/Grok/OpenCode/Antigravity, and local/remote/relay/tunnel all converge on the same environment-local worker for these effects. No transport, contract, provider adapter, client layout, empty/loading/error view, theme, width, or platform rendering changes here. One available real provider is sufficient for app recordings; provider availability must be checked without changing the intended hub configuration.

### Remaining execution and infrastructure branches, and how they collapse

| Branch | Mapping / recording requirement | Capturability and status |
| --- | --- | --- |
| Stop executor succeeds before deadline; success settlement accepted | R3; R4 additionally proves success on a later attempt. | Ready, required recordings above. |
| Stop executor fails with own typed error before deadline | R5 preserves original cause. Repeating that error through attempt 5 is an optional R5 variant; terminalization is already exercised by R1/R2. | Ready, controlled executor error harness/app injection. No separate minimum recording. |
| Stop executor dies or self-interrupts before deadline | `Effect.exit` routes its unchanged Cause through warning → retry/fail; stop never matches the provider-interrupt-only “already inactive” special case. Optional R5 variant emits a unique defect/self-interrupt cause. | Ready as labelled harness; source inventory sufficient for minimum. A self-interrupt outcome must not be narrated as durable cancellation. |
| Timeout occurs with attempts remaining / at exhaustion | R1/R2 must retain all claims, errors, available-at backoffs and final fail; R4 proves retry deadline reset. Default maxAttempts 5; custom values share the same threshold branch. | Ready, required. No need to override production maxAttempts. |
| Durable cancellation wins active race, or commits before success/retry/fail write | R7 represents active cancellation. Settlement writes require running status and owner; a cancelled row cannot be revived. Optional R7 barrier variants can cancel between executor exit and settlement, or during pending retry; validate the same cancelled terminal state. | Ready as labelled harness; no extra minimum recording. Deadline/cancellation ties have winner-dependent logs but must not resurrect cancelled rows. |
| No claim / failed claim / pre-execution durable-read error | No claim means wrapper not entered; claim/read errors retain existing worker handling (including replay-safe claim requeue). R0/R6 source inventory; optional service-layer fault injection. | Ready as labelled harness; outside changed execution outcome. |
| Success settlement or retry/fail write throws; lost ownership | Existing replay-safe success-settlement errors requeue a stop; error-settlement failures follow existing terminalize/requeue handlers. Lost lease yields a worker error unless durable cancellation explains it. Non-stop process-bound success-settlement failures terminalize rather than replay. | Ready as labelled outbox-fault harness; unchanged infrastructure branches, not additional minimum captures. No claim that timeout bounds DB reads/writes or settlement handlers. |
| Worker scope interrupted / process ends while executing | Process-loss replay is R2. Normal shutdown/fiber interruption is not a durable cancel request; interruption and startup/shutdown reconciliation must be distinguished from timeout settlement. Optional controlled fiber-interruption harness or owned-server SIGTERM transcript. | Ready; process-loss minimum R2. Label cleanup transitions separately in every recording. |
| Interruption-resistant executor/finalizer or synchronously blocked event loop | Timeout may not settle at 30s. This is the explicit limit in finding 3, not a promised recovery row. Optional harness uses a separately releasable finalizer/uninterruptible gate and records delayed settlement; a synchronous block needs a separately owned process. | Capturable as labelled limit demonstration; source inventory sufficient. Never run such a fixture in the shared server. |
| Exact deadline race (success/error/cancellation versus timer) | Whichever completion wins may determine error/log identity; no new deterministic semantic state beyond R3/R5/R7/R1. Use ≥5s margins in minimum fixtures instead of claiming an exact 30.000s tie outcome. | Capturable with controlled barriers; no extra minimum recording. |

## Capture contract for the next pass

- Run the same row driver against the exact requested base and head, isolated synthetic state per side, using each revision's real server assembly. Every run creates a new UTC-stamped directory under `/tmp/lead-25675-r2/proof-audit-1/` (or a separately authorized capture directory); never overwrite a previous recording. Drivers, logs, downloaded inputs, manifests and archives stay under the single scratch root. Code checkouts must be separately owned sibling worktrees, never this reviewed worktree or another lead's checkout. Do not execute the downloaded old scripts unchanged: they contain older revisions, paths and allowlists.
- Real app bootstrap creates/migrates a fresh DB with `T3CODE_DEV_SEED=0`; use authenticated RPC to create project and thread. No live state writes. For stable hangs/errors, seed a pending effect only with the owned server stopped, and label both seed and exact executor-only fault injection. Preserve immutable source SHA, injected diff, source hash and worker settings. Do not shorten 30s, reduce maxAttempts, replace the worker, or present a synthetic provider-start executor as a model reply.
- Record UTC action times, owned PIDs, process/recovery/cleanup boundaries, command/thread/run/effect IDs, request type, every outbox INSERT/UPDATE (`status`, attempt count, available/created/updated/completed times, lease owner/expiry, full last_error), executor entry/interruption/finalization, worker warnings and provider/message events from `orchestration_events`. Persisted retry delay is `available_at - updated_at`, not the variable wall time until the daemon next claims.
- Snapshot-only sampling can miss 100ms retry states. An append-only audit trigger in the synthetic DB or equally lossless transition recorder is required. Use read-only observers. Record actual stop-terminal transition before later provider-start claim, and real completed reply/run plus settled relevant outbox for app rows. R7/R8 explicitly contain stubbed successors and have no real-provider reply assertion.
- Before baselines for R1/R2/R4 stay alive through their full observation window. Requeue/cancellation caused by subsequent SIGTERM cleanup is not successful live recovery. Finalizers may add scheduling overhead, so show actual attempt times rather than promise 30s total or an exact aggregate duration.
- Retain raw and normalized transcripts, normalization rules, exact environment/model/tool versions, invocation and seed/injection source, and selected-run manifest. Redact credentials and startup pairing links. Upload evidence before deleting anything. Stop only recorded owned PIDs. Check free disk during capture; under 5 GB stops the run.

## Audit coverage and friction

Covered: exact requested diff and local head, production worker/executor/daemon, both stop-effect producers, recursive service and MCP bypass, outbox claim/lanes/retry/settlement/cancellation/process-loss behavior, both added regression variants and existing cancellation regressions, runtime startup, vendored Effect timeout implementation, initial PR metadata/status/comments/reviews, previous index and rebuild/app/injection/worker-harness scripts. No test execution or runtime capture was required for this pass-1 derivation. No server or child process was started that needs cleanup. Worktree started and ended clean. Free disk observed: 31 GB.

FRICTION:
- forge | Initial PR metadata/status/comments/reviews succeeded, but later `cafe pr view` retries returned retryable `SERVER_ERROR` / `ConnectionRefused`; the latest refresh could not be saved. | A stable GitCafe endpoint or a retained initial metadata snapshot would avoid the refresh gap; initial response and local immutable revisions identify the review target.
- infra | One evidence-script download failed with `curl: (6) Could not resolve host`; an IPv4 retry succeeded. | Reliable DNS or explicit IPv4 on this host would avoid the interrupted download.

VERDICT: SUFFICIENT