# RPC fallback evidence, PR 16874

Linux real-app RPC against controlled POSIX ps and failing native monitor. Windows is unverified. Source is pinned and clean before and after each case. Historical matrix.md is obsolete and is not evidence.

| Round | SHA | Case | Window starts at 0/10/20/30/40/50/60s | Peak live | Busy labels | Start gaps (s) | Completion-to-next-start (s) | Valid |
|---|---|---|---|---|---|---|---|---|
| after | a77f9be73aec486def0b419b80431f9a853a423c | fast | 1/3/4/4/5/5/5 | 1 | 1 | 2.078, 4.070, 8.059, 16.078 | 2.057, 4.050, 8.039, 16.058 | True |
| after | a77f9be73aec486def0b419b80431f9a853a423c | slow | 1/1/2/3/4/4/5 | 1 | 1 | 13.079, 13.385, 12.754, 18.559 | 10.579, 10.884, 10.253, 16.058 | True |
| after | a77f9be73aec486def0b419b80431f9a853a423c | many-fast | 1/2/3/4/4/4/4 | 1 | 20 | 4.081, 8.074, 16.081 | 4.061, 8.053, 16.060 | True |
| after | a77f9be73aec486def0b419b80431f9a853a423c | many-slow | 1/1/2/3/3/4/4 | 1 | 20 | 12.820, 13.044, 18.553 | 10.319, 10.543, 16.053 | True |
| after | a77f9be73aec486def0b419b80431f9a853a423c | hung | 1/1/1/1/1/1/1 | 1 | 20 | 75.940 | none | True |
| after | a77f9be73aec486def0b419b80431f9a853a423c | close | 1/2/3/4/4/4/4 | 1 | 2 | 4.101, 8.073, 16.051 | 4.060, 8.072, 16.051 | False |
| before | 5a969c17e76301e45e53e1fd9aad2f5b4599d865 | fast | 1/3/4/4/5/5/5 | 1 | 1 | 2.075, 4.075, 8.072, 16.126 | 2.055, 4.055, 8.052, 16.105 | True |
| before | 5a969c17e76301e45e53e1fd9aad2f5b4599d865 | slow | 1/3/4/4/5/5/5 | 1 | 0 | 3.022, 5.020, 9.024, 17.016 | none | True |
| before | 5a969c17e76301e45e53e1fd9aad2f5b4599d865 | many-fast | 1/2/3/4/4/4/4 | 1 | 20 | 4.073, 8.080, 16.078 | 4.053, 8.059, 16.058 | True |
| before | 5a969c17e76301e45e53e1fd9aad2f5b4599d865 | many-slow | 1/2/3/3/4/4/4 | 1 | 0 | 5.014, 9.082, 16.982 | none | True |
| before | 5a969c17e76301e45e53e1fd9aad2f5b4599d865 | hung | 1/1/2/3/3/3/3 | 1 | 20 | 10.015, 18.022, 34.030, 62.080 | none | True |
| before | 5a969c17e76301e45e53e1fd9aad2f5b4599d865 | close | 1/2/3/4/4/4/4 | 1 | 2 | 4.068, 8.046, 16.046 | 4.047, 8.045, 16.046 | False |

Expected busy labels are ev-case-N for actual sleep children verified under actual terminal shells through /proc. The table counts terminals with nonempty observed busy labels; full labels are retained in measured-matrix.json. Every query START/END, metadata upsert/removal, close reply, and PID-identity sampling event is retained in sanitized archives. 20ms queries can fall between 50ms /proc samples; START/END remain authoritative.

Native failure causes snapshotSucceeded=false even when fallback succeeds, so exponential backoff also applies to fast fallback. AFTER additionally paces at 4x measured fallback query duration (including execution overhead), capped at 60s. Poll timeout is 15s; closeIdle keeps POSIX 1s / Windows 1.5s by source inspection, not by this terminal.close case. Two unchanged idle snapshots enable quiet-terminal skipping every 30 ticks; the busy fixtures prevent that from explaining counts, and failed snapshot acquisition clears quiet state.

## Hung and close details

### after hung
Directory: /tmp/ev-16874/20261025-004930-after-hung
Counts through full window: [{"second": 0, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": -11.805997}, {"second": 10, "cumulativeStarts": 3, "windowStarts": 1, "live": [1107086], "sampleDeltaMs": -10.600645}, {"second": 20, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": 6.345878}, {"second": 30, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": -7.837772}, {"second": 40, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": 24.607845}, {"second": 50, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": 3.181302}, {"second": 60, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": -7.053776}, {"second": 70, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": 11.221529}, {"second": 80, "cumulativeStarts": 4, "windowStarts": 2, "live": [1107422], "sampleDeltaMs": 8.498322}, {"second": 90, "cumulativeStarts": 4, "windowStarts": 2, "live": [1107422], "sampleDeltaMs": -2.480672}, {"second": 100, "cumulativeStarts": 4, "windowStarts": 2, "live": [], "sampleDeltaMs": -16.125361}, {"second": 110, "cumulativeStarts": 4, "windowStarts": 2, "live": [], "sampleDeltaMs": -20.850794}, {"second": 120, "cumulativeStarts": 4, "windowStarts": 2, "live": [], "sampleDeltaMs": 10.799672}, {"second": 130, "cumulativeStarts": 4, "windowStarts": 2, "live": [], "sampleDeltaMs": -3.501939}, {"second": 140, "cumulativeStarts": 4, "windowStarts": 2, "live": [], "sampleDeltaMs": -16.081493}, {"second": 150, "cumulativeStarts": 4, "windowStarts": 2, "live": [], "sampleDeltaMs": 20.126121}]
Queries: [{"pid": 1106896, "mode": ".02", "disappearanceSeconds": 0.067229177, "durationSeconds": 0.020452157, "termSignals": 0}, {"pid": 1107080, "mode": ".02", "disappearanceSeconds": 0.067617896, "durationSeconds": 0.020386904, "termSignals": 0}, {"pid": 1107086, "mode": "hung", "disappearanceSeconds": 15.890138239, "durationSeconds": null, "termSignals": 1}, {"pid": 1107422, "mode": "hung", "disappearanceSeconds": 15.986684111, "durationSeconds": null, "termSignals": 1}]
Disappearance-to-next-start seconds (50ms observation resolution): [60.049720378]
Close replies: []
Late closed-terminal activity: []
Cleanup replies: [{"terminalId": "case-1", "elapsedMs": 11.568742, "event": "cleanup-close-reply", "ns": "343843883834727"}, {"terminalId": "case-2", "elapsedMs": 5.345248, "event": "cleanup-close-reply", "ns": "343843889227503"}, {"terminalId": "case-3", "elapsedMs": 4.202964, "event": "cleanup-close-reply", "ns": "343843893631404"}, {"terminalId": "case-4", "elapsedMs": 3.5078, "event": "cleanup-close-reply", "ns": "343843897164743"}, {"terminalId": "case-5", "elapsedMs": 3.4917, "event": "cleanup-close-reply", "ns": "343843900729600"}, {"terminalId": "case-6", "elapsedMs": 3.599953, "event": "cleanup-close-reply", "ns": "343843904356183"}, {"terminalId": "case-7", "elapsedMs": 6.721489, "event": "cleanup-close-reply", "ns": "343843911102048"}, {"terminalId": "case-8", "elapsedMs": 4.571526, "event": "cleanup-close-reply", "ns": "343843915698851"}, {"terminalId": "case-9", "elapsedMs": 3.602489, "event": "cleanup-close-reply", "ns": "343843919326957"}, {"terminalId": "case-10", "elapsedMs": 3.915715, "event": "cleanup-close-reply", "ns": "343843923299199"}, {"terminalId": "case-11", "elapsedMs": 3.202988, "event": "cleanup-close-reply", "ns": "343843926527385"}, {"terminalId": "case-12", "elapsedMs": 3.31065, "event": "cleanup-close-reply", "ns": "343843929860427"}, {"terminalId": "case-13", "elapsedMs": 3.905717, "event": "cleanup-close-reply", "ns": "343843933788636"}, {"terminalId": "case-14", "elapsedMs": 2.912023, "event": "cleanup-close-reply", "ns": "343843936750212"}, {"terminalId": "case-15", "elapsedMs": 3.209682, "event": "cleanup-close-reply", "ns": "343843939984619"}, {"terminalId": "case-16", "elapsedMs": 3.506989, "event": "cleanup-close-reply", "ns": "343843943541471"}, {"terminalId": "case-17", "elapsedMs": 2.57495, "event": "cleanup-close-reply", "ns": "343843946137892"}, {"terminalId": "case-18", "elapsedMs": 3.157342, "event": "cleanup-close-reply", "ns": "343843949316595"}, {"terminalId": "case-19", "elapsedMs": 3.298107, "event": "cleanup-close-reply", "ns": "343843952636413"}, {"terminalId": "case-20", "elapsedMs": 2.607682, "event": "cleanup-close-reply", "ns": "343843955266787"}]

### after close
Directory: /tmp/ev-16874/20261025-005214-after-close
Counts through full window: [{"second": 0, "cumulativeStarts": 2, "windowStarts": 1, "live": [], "sampleDeltaMs": -11.758638}, {"second": 10, "cumulativeStarts": 3, "windowStarts": 2, "live": [], "sampleDeltaMs": 18.884913}, {"second": 20, "cumulativeStarts": 4, "windowStarts": 3, "live": [], "sampleDeltaMs": 19.281203}, {"second": 30, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": 11.740168}, {"second": 40, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": -3.730742}, {"second": 50, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": -4.843489}, {"second": 60, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": -11.378847}]
Queries: [{"pid": 1108289, "mode": ".02", "disappearanceSeconds": 0.037079048, "durationSeconds": 0.02051215, "termSignals": 0}, {"pid": 1108291, "mode": "gate", "disappearanceSeconds": 0.08970526, "durationSeconds": 0.040895817, "termSignals": 0}, {"pid": 1108298, "mode": "gate", "disappearanceSeconds": 0.040838359, "durationSeconds": 0.00030362, "termSignals": 0}, {"pid": 1108312, "mode": "gate", "disappearanceSeconds": 0.032071102, "durationSeconds": 0.000317806, "termSignals": 0}, {"pid": 1108377, "mode": "gate", "disappearanceSeconds": 0.009993978, "durationSeconds": 0.000236604, "termSignals": 0}]
Disappearance-to-next-start seconds (50ms observation resolution): [4.011476401, 8.031890189, 16.019306629]
Close replies: [{"terminalId": "case-1", "elapsedMs": 10.595195, "event": "close-reply", "ns": "343853903631936"}]
Late closed-terminal activity: []
Cleanup replies: [{"terminalId": "case-1", "elapsedMs": 3.960009, "event": "cleanup-close-reply", "ns": "343913875835322"}, {"terminalId": "case-2", "elapsedMs": 6.019302, "event": "cleanup-close-reply", "ns": "343913881901351"}]

### before hung
Directory: /tmp/ev-16874/20261025-005837-before-hung
Counts through full window: [{"second": 0, "cumulativeStarts": 3, "windowStarts": 1, "live": [1114492], "sampleDeltaMs": 17.154931}, {"second": 10, "cumulativeStarts": 3, "windowStarts": 1, "live": [], "sampleDeltaMs": -15.857796}, {"second": 20, "cumulativeStarts": 4, "windowStarts": 2, "live": [], "sampleDeltaMs": 11.508775}, {"second": 30, "cumulativeStarts": 5, "windowStarts": 3, "live": [1114522], "sampleDeltaMs": -3.562645}, {"second": 40, "cumulativeStarts": 5, "windowStarts": 3, "live": [], "sampleDeltaMs": -9.109448}, {"second": 50, "cumulativeStarts": 5, "windowStarts": 3, "live": [], "sampleDeltaMs": -22.066353}, {"second": 60, "cumulativeStarts": 5, "windowStarts": 3, "live": [], "sampleDeltaMs": 10.383225}, {"second": 70, "cumulativeStarts": 6, "windowStarts": 4, "live": [], "sampleDeltaMs": 13.210891}, {"second": 80, "cumulativeStarts": 6, "windowStarts": 4, "live": [], "sampleDeltaMs": 24.74518}, {"second": 90, "cumulativeStarts": 6, "windowStarts": 4, "live": [], "sampleDeltaMs": 16.02297}, {"second": 100, "cumulativeStarts": 6, "windowStarts": 4, "live": [], "sampleDeltaMs": 9.118714}, {"second": 110, "cumulativeStarts": 6, "windowStarts": 4, "live": [], "sampleDeltaMs": 8.990977}, {"second": 120, "cumulativeStarts": 6, "windowStarts": 4, "live": [], "sampleDeltaMs": 2.612263}, {"second": 130, "cumulativeStarts": 7, "windowStarts": 5, "live": [], "sampleDeltaMs": 1.486201}, {"second": 140, "cumulativeStarts": 7, "windowStarts": 5, "live": [], "sampleDeltaMs": 3.257176}, {"second": 150, "cumulativeStarts": 7, "windowStarts": 5, "live": [], "sampleDeltaMs": 9.608422}]
Queries: [{"pid": 1114442, "mode": ".02", "disappearanceSeconds": 0.057403234, "durationSeconds": 0.020602097, "termSignals": 0}, {"pid": 1114490, "mode": ".02", "disappearanceSeconds": 0.054163967, "durationSeconds": 0.02036852, "termSignals": 0}, {"pid": 1114492, "mode": "hung", "disappearanceSeconds": 1.994001714, "durationSeconds": null, "termSignals": 1}, {"pid": 1114502, "mode": "hung", "disappearanceSeconds": 2.000670684, "durationSeconds": null, "termSignals": 1}, {"pid": 1114522, "mode": "hung", "disappearanceSeconds": 2.010397191, "durationSeconds": null, "termSignals": 1}, {"pid": 1114556, "mode": "hung", "disappearanceSeconds": 2.006597712, "durationSeconds": null, "termSignals": 1}, {"pid": 1114864, "mode": "hung", "disappearanceSeconds": 1.99805091, "durationSeconds": null, "termSignals": 1}]
Disappearance-to-next-start seconds (50ms observation resolution): [8.020502597, 16.021550681, 32.019928209, 60.073687658]
Close replies: []
Late closed-terminal activity: []
Cleanup replies: [{"terminalId": "case-1", "elapsedMs": 17.684695, "event": "cleanup-close-reply", "ns": "344389949201379"}, {"terminalId": "case-2", "elapsedMs": 4.966978, "event": "cleanup-close-reply", "ns": "344389954245602"}, {"terminalId": "case-3", "elapsedMs": 5.801713, "event": "cleanup-close-reply", "ns": "344389960076510"}, {"terminalId": "case-4", "elapsedMs": 6.904662, "event": "cleanup-close-reply", "ns": "344389967010157"}, {"terminalId": "case-5", "elapsedMs": 5.440756, "event": "cleanup-close-reply", "ns": "344389972482111"}, {"terminalId": "case-6", "elapsedMs": 6.383234, "event": "cleanup-close-reply", "ns": "344389978909609"}, {"terminalId": "case-7", "elapsedMs": 4.736755, "event": "cleanup-close-reply", "ns": "344389983673255"}, {"terminalId": "case-8", "elapsedMs": 6.966287, "event": "cleanup-close-reply", "ns": "344389990675841"}, {"terminalId": "case-9", "elapsedMs": 4.63715, "event": "cleanup-close-reply", "ns": "344389995338899"}, {"terminalId": "case-10", "elapsedMs": 3.768861, "event": "cleanup-close-reply", "ns": "344389999131584"}, {"terminalId": "case-11", "elapsedMs": 4.96242, "event": "cleanup-close-reply", "ns": "344390004118609"}, {"terminalId": "case-12", "elapsedMs": 3.350826, "event": "cleanup-close-reply", "ns": "344390007512035"}, {"terminalId": "case-13", "elapsedMs": 3.261227, "event": "cleanup-close-reply", "ns": "344390010803330"}, {"terminalId": "case-14", "elapsedMs": 4.402578, "event": "cleanup-close-reply", "ns": "344390015277432"}, {"terminalId": "case-15", "elapsedMs": 3.564988, "event": "cleanup-close-reply", "ns": "344390018875161"}, {"terminalId": "case-16", "elapsedMs": 3.083344, "event": "cleanup-close-reply", "ns": "344390021984304"}, {"terminalId": "case-17", "elapsedMs": 5.074489, "event": "cleanup-close-reply", "ns": "344390027083279"}, {"terminalId": "case-18", "elapsedMs": 3.547695, "event": "cleanup-close-reply", "ns": "344390030654558"}, {"terminalId": "case-19", "elapsedMs": 3.326911, "event": "cleanup-close-reply", "ns": "344390034004423"}, {"terminalId": "case-20", "elapsedMs": 4.263738, "event": "cleanup-close-reply", "ns": "344390038300171"}]

### before close
Directory: /tmp/ev-16874/20261025-010120-before-close
Counts through full window: [{"second": 0, "cumulativeStarts": 2, "windowStarts": 1, "live": [1115445], "sampleDeltaMs": 2.38304}, {"second": 10, "cumulativeStarts": 3, "windowStarts": 2, "live": [], "sampleDeltaMs": 17.913135}, {"second": 20, "cumulativeStarts": 4, "windowStarts": 3, "live": [], "sampleDeltaMs": -3.699918}, {"second": 30, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": 22.736206}, {"second": 40, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": -2.970625}, {"second": 50, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": 20.03303}, {"second": 60, "cumulativeStarts": 5, "windowStarts": 4, "live": [], "sampleDeltaMs": 2.879844}]
Queries: [{"pid": 1115443, "mode": ".02", "disappearanceSeconds": 0.060428218, "durationSeconds": 0.020442789, "termSignals": 0}, {"pid": 1115445, "mode": "gate", "disappearanceSeconds": 0.052938087, "durationSeconds": 0.020943598, "termSignals": 0}, {"pid": 1115455, "mode": "gate", "disappearanceSeconds": 0.027731913, "durationSeconds": 0.000308128, "termSignals": 0}, {"pid": 1115459, "mode": "gate", "disappearanceSeconds": 0.031993557, "durationSeconds": 0.000231574, "termSignals": 0}, {"pid": 1115487, "mode": "gate", "disappearanceSeconds": 0.039302429, "durationSeconds": 0.000260329, "termSignals": 0}]
Disappearance-to-next-start seconds (50ms observation resolution): [4.015377149, 8.017855932, 16.014292552]
Close replies: [{"terminalId": "case-1", "elapsedMs": 9.561296, "event": "close-reply", "ns": "344399752254770"}]
Late closed-terminal activity: []
Cleanup replies: [{"terminalId": "case-1", "elapsedMs": 3.660307, "event": "cleanup-close-reply", "ns": "344459744269983"}, {"terminalId": "case-2", "elapsedMs": 4.388263, "event": "cleanup-close-reply", "ns": "344459748693421"}]

## Reproduction and cleanup

Unpack reproduction-scripts.tar.gz into /tmp/ev-16874. Requirements: prepared /home/ubuntu/wt/pr-16874 checkout with installed dependencies, Linux unshare user/mount namespaces, Python 3, Node, real app build dependencies. The runner never checks out source. Parent must coordinate exact SHA first.

```sh
python3 /tmp/ev-16874/round.py --sha a77f9be73aec486def0b419b80431f9a853a423c --label after
python3 /tmp/ev-16874/round.py --sha 5a969c17e76301e45e53e1fd9aad2f5b4599d865 --label before
```

Auditor one-row reproduction: coordinate AFTER checkout, then run `python3 /tmp/ev-16874/run.py --sha a77f9be73aec486def0b419b80431f9a853a423c --label audit-slow --mode slow --terminals 1 --seconds 60`. Read README.md for mechanism and limits.

Each case closes terminals via RPC, stops captured server/observer/driver PIDs, checks original process start identities, and records fixtureSurvivors/terminalSurvivors and source stability in result.json. No broad kill patterns. Retain scripts until audit; cleanup paths are the exact directory values in measured-matrix.json, plus reproduction archive and summary artifacts under /tmp/ev-16874. Private scratch data and raw server logs must never be uploaded.

Friction category|event|prevention
expectations|native-failure backoff applies even to successful fast fallback|measure selected production source rather than infer cadence
credentials|startup logs contain pairing URLs|explicit evidence allowlist and secret scan
coordination|before capture requires exclusive checkout switch|parent readiness handshake and exact SHA/source guards

## Original close cleanup failure

Both original close captures are invalid for cleanup: BEFORE child PID 1115437 / startTicks 34439740 and AFTER child PID 1108281 / startTicks 34385161 survived. Both exact identities were subsequently terminated and checked no longer live; manual-cleanup.json is included. Original result.json files remain unchanged. Close reply was 9.561ms BEFORE and 10.595ms AFTER, with metadata removal and no late activity in both. This is present on baseline, not introduced by this change. Supplemental terminal-owned fixture evidence will be separate.

# Supplemental close-mid-query evidence

Original background-job close failures remain preserved in the original matrix/archive. This supplement uses a separate fixture with `set +m` and an EXIT/HUP/TERM trap that terminates its captured child. It retains the same real-app RPC, child identity verification, busy labels, gated ps query, metadata removal, and one-minute observation. No source changes.

| Round | Source SHA | Close reply ms | Late closed activity | Query peak live | Valid cleanup |
|---|---|---|---|---|---|
| supplement-after-close | cca46259f59d82f31c7277cc2129b4a7185df890 | 9.046191 | 0 | 1 | True |
| supplement-before-close | 5a969c17e76301e45e53e1fd9aad2f5b4599d865 | 12.678534 | 0 | 1 | True |

Both terminals initially reported their exact ev-case-N busy labels. Closed terminal metadata was removed before releasing the in-flight query gate. The surviving terminal remained observable. Every captured query and terminal child was gone at cleanup; source guards remained stable. These terminal.close results do not independently verify closeIdle timeout budgets or Windows.

Reproduce on a parent-coordinated clean checkout:

```sh
python3 /tmp/ev-16874/supplement-run.py --sha cca46259f59d82f31c7277cc2129b4a7185df890 --label supplement-after-close --mode close --terminals 2 --seconds 60
python3 /tmp/ev-16874/supplement-run.py --sha 5a969c17e76301e45e53e1fd9aad2f5b4599d865 --label supplement-before-close --mode close --terminals 2 --seconds 60
```

The original AFTER full round uses a77f9be73aec486def0b419b80431f9a853a423c; supplemental AFTER uses merged cca46259f59d82f31c7277cc2129b4a7185df890. The parent reports no Manager/processRunner implementation differences from the merge. Evidence records both actual SHAs explicitly.

Cleanup verification checked 349 recorded server/query/fixture identities across fourteen captures. Original orphaned children were explicitly cleaned by recorded PID+startTicks; invalid original results remain untouched. All other captures cleaned automatically. Keep /tmp/ev-16874 scripts until independent audit. Cleanup targets are exact case directories in measured-matrix.json and supplement-close.json; never upload their private/, home/, or server.log.

Friction category|event|prevention
fixture lifecycle|background shell job survived close on both baseline and after|separate supplemental fixture disables job control and traps child cleanup; preserve original failures
measurement|metadata map contains empty label arrays|count only observed nonempty labels
coordination|independent auditor needs a stopped server and pinned source|explicit completion handshake before source switch