# Server runtime evidence — [#33587](https://git.cafe/pingdotgg/t3code/pull/33587)

Observed scope beyond the PR title: agent-access-off also gains orphan cleanup; cached tools with no daemon record skip the stop subprocess; dead-PID stale records are removed without a subprocess; recorded-version/fallback CLI selection is exercised. Damaged/unreadable records now cause a best-effort CLI attempt but still leave the daemon alive. These are intended or explicitly limited behaviors, not a claim that every orphan is recoverable.

Fresh captures: **10 of 12 matrix rows**, covering **all 10 rows requested for new capture**, with 17 fixture variants on each side (34 runs). R02 is carried forward from existing evidence; R12 is unit-level and has no capture requirement. No required runtime row is unverified.

Environment: Linux 6.1.102 x86_64; Node v26.12.1; agent-device current 0.21.31 and older 0.21.30; expo-device-hub 0.19.1; Android emulator package 37.2.13 (capability prerequisite only, no emulator booted), ADB 37.0.1 with a fresh isolated server/socket per RPC fixture. Real server started with `node apps/server/src/bin.ts serve --base-dir <fresh scratch base> --port <unused> --host 127.0.0.1 --no-browser`, `T3CODE_DEV_SEED=0`. No live database was used, no project or provider turn was created, and captures contain synthetic fixture data only.

After ran first at clean [9f3180ba42](https://git.cafe/pingdotgg/t3code/commit/9f3180ba42ff5597d9a3d8a2f62ff41ce52bab92). Before used that same checkout with exactly LocalDeviceHost.ts and DeviceToolchain.ts replaced by [9110dc2c93](https://git.cafe/pingdotgg/t3code/commit/9110dc2c932cb3bc57dc2845b9f1ea362f248796) bytes. Every run prints both actual SHA256 values and asserts equality against its required Git blob. No other runtime server file differs between the two revisions. The same capture driver was used for all published runs (SHA256 `73d4ddae434db385028af1998d74f2da19868185ec5df4aee58cb383d0952dde`).

Observation: NODE_OPTIONS preload wraps actual child_process spawn/execFile/fork calls and syncs ESM builtins; each action transcript includes every observed agent-device process and all npm/pnpm invocations during its window. Process identity checks use /proc PID start ticks and exclude zombies. RPC uses the real browser-session exchange and an RPC ticket in `wsTicket`; all credentials are redacted. Real hub HTTP responses and a post-action inspect-only RPC establish continued service for applicable rows. Missing stored boolean settings mean their default false; RPC responses also report agentAccessEnabled explicitly.

| Row | Fixture | Before ([9110dc2c93](https://git.cafe/pingdotgg/t3code/commit/9110dc2c932cb3bc57dc2845b9f1ea362f248796)) link + outcome | After ([9f3180ba42](https://git.cafe/pingdotgg/t3code/commit/9f3180ba42ff5597d9a3d8a2f62ff41ce52bab92)) link + outcome | Verdict |
|---|---|---|---|---|
| R01 | No record: shutdown and agent-off; both repeated after agent tools were cached and the record disappeared. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/25dc5b03c0db6f9b/R01-before.txt): Uncached: no CLI, no daemon. Cached: current stop CLI attempted, record stays absent, daemon remained alive. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/bc57c4361819b37f/R01-after.txt): All four variants skip the stop CLI; missing-record cached daemon remains alive. | PASS: new guard avoids the cached-state subprocess; hub/server lifecycles preserved. |
| R03 | Live orphan 0.21.30; old and current CLI entries and matching sentinels installed. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/a01d7f3d320dc90d/R03-before.txt): No stop CLI; old daemon and record survive. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/8f9adb48f3390c3d/R03-after.txt): Recorded 0.21.30 CLI selected; daemon dead and record removed. | PASS: recorded-version preference and real termination observed. |
| R04 | Live orphan; old sentinel mismatched, or version omitted, or version empty. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/cea0f75036356814/R04-before.txt): No stop CLI; daemon and record survive in all three variants. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/804abdefbfe34b57/R04-after.txt): Current 0.21.31 CLI selected in all three; daemon dead and record removed. | PASS: fallback also retires the real older daemon. |
| R05 | Live older orphan; both installed CLI sentinels mismatched. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/c5696cb1f005f082/R05-before.txt): No CLI/install; daemon and record survive. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/d9497b9e846bac04/R05-after.txt): No CLI/install; daemon and record survive. | PASS for best-effort return; cleanup intentionally unavailable. |
| R06 | Live daemon with malformed JSON, schema-invalid record, or mode-000 unreadable record. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/9689ea201f7e1f5a/R06-before.txt): No stop CLI; daemon and damaged/unreadable record survive. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/8be7a61a76cabb25/R06-after.txt): Current CLI attempted; daemon and record still survive. Shutdown completes in each variant. | PASS for failure containment; daemon retirement is not achieved. |
| R07 | Real daemon terminated first; its valid registration restored as a stale record. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/4463ae6a00e7ef3c/R07-before.txt): No stop CLI; stale record remains; no live daemon. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/60b1a7a574eb56e2/R07-after.txt): No stop CLI; stale record removed; no live daemon. | PASS: updated dead-PID fast path observed. |
| R08 | This server starts hub and daemon via authenticated device.configure, then SIGTERM. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/0e3f20d2c2729202/R08-before.txt): Current CLI stops daemon; daemon and hub records removed; helpers and server exit. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/9236bd5b5d48cbc8/R08-after.txt): Same observed helper and record cleanup. | PASS: existing working shutdown path preserved. |
| R09 | Live orphan; RPC enables hub without agent readiness, then disables agent access. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/492c7bd55eedd82e/R09-before.txt): No stop CLI; orphan and record survive. Hub/server continue serving. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/c73b917dfe98bf17/R09-after.txt): Current CLI stops orphan and removes record; hub/server continue serving. | PASS: orphan cleanup reaches stopAgent through real RPC. |
| R10 | This server starts hub and daemon; RPC disables only agent access. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/00bc0ee3e9a6139c/R10-before.txt): Current CLI stops daemon and removes record; hub/server remain serving. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/74bef0392abdc6d9/R10-after.txt): Same observed cleanup; agent access false and running agent version null. | PASS: working stopAgent path preserved. |
| R11 | This server starts hub and daemon; RPC disables device support and agent access. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/87cf972d6d471ea1/R11-before.txt): Both helpers stop and records disappear; server remains serving, state disabled. | [Transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/75101f021d430e2c/R11-after.txt): Same observed cleanup and state. | PASS: non-finalizer stop caller preserved. |
| R02 | Current-pin live orphan; existing evidence carried forward per supplied matrix. | [Existing before transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/d6b9565a3d5e14d3/issue-33552-before-main.txt): daemon survives shutdown. | [Existing after transcript](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/4840b006e5a01545/issue-33552-after-fix.txt): daemon stops and record disappears. | CARRIED FORWARD, no new capture; matrix notes legacy baseline/hash and recorded-version metadata qualifications. |
| R12 | Deterministic filesystem/Node/runner failure and timeout injection. | No runtime capture required. | No runtime capture required. | UNIT-LEVEL scope; no additional unit tests run by this evidence step. |

## Every fixture: before/after observed state and latency

Values are state immediately after the action, before explicit fixture cleanup. Stop column names the CLI version, or “none”. Latencies include shutdown completion or the configure RPC response; these are observations, not a benchmark.

| Fixture | Before daemon / record / stop CLI | After daemon / record / stop CLI | Before hub / server | After hub / server | Action ms before → after |
|---|---|---|---|---|---|
| R01-shutdown | absent/dead / absent / none | absent/dead / absent / none | off / off | off / off | 942 → 1968 |
| R01-agentoff | absent/dead / absent / none | absent/dead / absent / none | serving / serving | serving / serving | 797 → 410 |
| R01-cached-shutdown | alive / absent / 0.21.31 | alive / absent / none | off / off | off / off | 1517 → 992 |
| R01-cached-agentoff | alive / absent / 0.21.31 | alive / absent / none | serving / serving | serving / serving | 621 → 450 |
| R03-older | alive / present / none | absent/dead / absent / 0.21.30 | off / off | off / off | 932 → 1534 |
| R04-ineligible | alive / present / none | absent/dead / absent / 0.21.31 | off / off | off / off | 1302 → 1684 |
| R04-noversion | alive / present / none | absent/dead / absent / 0.21.31 | off / off | off / off | 861 → 1576 |
| R04-emptyversion | alive / present / none | absent/dead / absent / 0.21.31 | off / off | off / off | 993 → 1810 |
| R05-noeligible | alive / present / none | alive / present / none | off / off | off / off | 975 → 1120 |
| R06-malformed | alive / present / none | alive / present / 0.21.31 | off / off | off / off | 1245 → 1061 |
| R06-schema | alive / present / none | alive / present / 0.21.31 | off / off | off / off | 681 → 1177 |
| R06-unreadable | alive / present / none | alive / present / 0.21.31 | off / off | off / off | 1145 → 1641 |
| R07-dead | absent/dead / present / none | absent/dead / absent / none | off / off | off / off | 1581 → 888 |
| R08-started-shutdown | absent/dead / absent / 0.21.31 | absent/dead / absent / 0.21.31 | off / off | off / off | 1216 → 1554 |
| R09-orphan-agentoff | alive / present / none | absent/dead / absent / 0.21.31 | serving / serving | serving / serving | 702 → 930 |
| R10-started-agentoff | absent/dead / absent / 0.21.31 | absent/dead / absent / 0.21.31 | serving / serving | serving / serving | 879 → 867 |
| R11-supportoff | absent/dead / absent / 0.21.31 | absent/dead / absent / 0.21.31 | off / serving | off / serving | 377 → 441 |

## What else changed

- R01 cached variants: before launches one stop CLI even with no record; after launches none. Both leave the intentionally unregistered live daemon for explicit fixture cleanup. Uncached variants launch no stop CLI on either side.
- R03 and all R04 variants: an orphan that survives before is terminated after, its registration disappears, and the action gains the recorded or fallback stop CLI.
- R05: no eligibility means no CLI/install on either side; the live daemon and registration remain.
- R06 variants: after adds one current-pin CLI attempt. All three still retain the live daemon and damaged or unreadable record; server shutdown completes on both sides.
- R07: after deletes the stale dead-PID record; neither side launches a stop CLI or creates a replacement daemon.
- R09: after adds one stop CLI and retires the orphan while preserving the existing live hub and server.
- R08, R10 and R11: helper lifetime, registration cleanup, settings outcome and continued-service behavior are preserved. No failed RPC or changed final service behavior was observed.
- Every action has measured timing variation, shown above. Fresh run directories, timestamps, tokens (redacted), PIDs/start ticks, and ephemeral ports differ by construction. Daemon registration processStartTime/stateDir/httpPort/pid therefore differ too; tool versions/code signatures are held fixed per corresponding fixture.
- Startup provider-version checks can overlap an action. They are unrelated npm view queries, not tool installations; the complete subprocess lists retain them. No npm/pnpm install ran during any action. Per-fixture counts are below.

| Fixture | npm/pnpm invocations before → after | Tool install invocations before → after |
|---|---|---|
| R01-shutdown | 0 → 0 | 0 → 0 |
| R01-agentoff | 0 → 0 | 0 → 0 |
| R01-cached-shutdown | 0 → 0 | 0 → 0 |
| R01-cached-agentoff | 0 → 0 | 0 → 0 |
| R03-older | 0 → 0 | 0 → 0 |
| R04-ineligible | 0 → 0 | 0 → 0 |
| R04-noversion | 0 → 0 | 0 → 0 |
| R04-emptyversion | 0 → 0 | 0 → 0 |
| R05-noeligible | 0 → 0 | 0 → 0 |
| R06-malformed | 0 → 1 | 0 → 0 |
| R06-schema | 2 → 1 | 0 → 0 |
| R06-unreadable | 0 → 1 | 0 → 0 |
| R07-dead | 0 → 0 | 0 → 0 |
| R08-started-shutdown | 0 → 0 | 0 → 0 |
| R09-orphan-agentoff | 0 → 0 | 0 → 0 |
| R10-started-agentoff | 0 → 0 | 0 → 0 |
| R11-supportoff | 0 → 0 | 0 → 0 |

## Scripts

[capture-driver.mjs](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/6fc6c7b5d3fa39c3/capture-driver.mjs)
[spawn-observer.cjs](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/9f733631299439ab/spawn-observer.cjs)
[run-rows.mjs](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/c8abb724ba1e8666/run-rows.mjs)
[setup-tools.sh](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/2480629c73899319/setup-tools.sh)
[capture-before.sh](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/303446d780ab4997/capture-before.sh)
[restore-head.sh](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/ab7733d40050d0e0/restore-head.sh)
[assemble-evidence.py](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/dc32b6642d00c2fe/assemble-evidence.py)
[verify-evidence.py](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/9ac4d639b9e8b25d/verify-evidence.py)
[make-index.py](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/d9155c21f119c5b9/make-index.py)


## Limits and cleanup

Linux real-server evidence only. Native macOS/Electron Quit is unverified; it is outside this server-only capture brief. R12 deterministic fault/timeout injection was not part of the requested runtime captures. A process-local endpoint cache is not directly exposed as an RPC field; R10 proves the observable stopped daemon, null running agent version, false agent permission and usable hub/server, rather than claiming direct memory inspection.

All fixture servers, hubs, daemon processes, and isolated ADB servers were stopped by captured PIDs. Completed fixture base directories were removed after transcripts were written. The lead worktree was restored to clean HEAD [9f3180ba42](https://git.cafe/pingdotgg/t3code/commit/9f3180ba42ff5597d9a3d8a2f62ff41ce52bab92); no commits or PR changes were made. A final audit of 1006 captured spawn PIDs found none still running. Remaining capture scratch is removed after upload. Disk stayed above the 5 GB stop threshold.

AI-assisted; verified with the tests and tools listed above.

FRICTION:
- tool | inherited RPC driver used ticket rather than wsTicket; readiness also queried a nonexistent descriptor path | keep a versioned capture driver aligned with the current client transport.
- infra | VM lacked the emulator package required to pass the server capability gate | provision SDK prerequisites or document them in the server capture runbook; resolved with an isolated scratch install.
- tool | ADB refused a hostname in its -L listener address | use -L tcp:<port> and a separate client ADB_SERVER_SOCKET.
- tool | a long batch received SIGTERM before the R08 action; the incomplete run is excluded and known children were gone | run bounded batches; remaining rows were recaptured successfully.