# PR #22193 fixture matrix: "fix(web): keep saved default model when changing effort"

Head `1ce968ca29`, before = merge-base `3b95e7831b`. Proof auditor, pass 1. Read-only.

## What the diff actually changes (from reading the code)

1. **Settings, General, "New threads", Model row** (`ProjectDefaultsSettings.tsx`). Before: the row showed `resolveDefaultProviderModelSelection(providers, saved)`. When the saved provider is not *selectable* (`enabled && isAvailable` on the server snapshot, `providerInstances.ts:228-243`), that function **swaps in another ready provider and its default model**. The TraitsPicker then wrote `createModelSelection(substitute.instanceId, substitute.model, options)`, which is the bug. After: `resolveDefaultModelSettingsPresentation` keeps the saved selection when the saved provider is unusable (turned off by settings or project override, `availability:"unavailable"`, or missing from the snapshot) or was substituted. It sets `effortLocked: true` (the **TraitsPicker is hidden**) and puts a status note on the row:
   - `"<DisplayName> is turned off."`
   - `"<DisplayName> is unavailable."`
   - `"This provider is not available on this environment."` (the instance is missing from the snapshot)
   - The row also renders when `activeEntry` is missing; before, it showed "No providers available" in that case.
2. **ProviderModelPicker trigger label** (shared by the composer and every Settings picker). When the active instance is off or unavailable and the model is not in the option list, the label now shows the saved model (catalog name, or the raw slug) instead of the **first option of that instance**. OpenCode and Antigravity are excluded.
3. **Composer effort is NOT touched.** Composer effort and model changes go to the composer draft and the sticky store (`useComposerSelection.ts:210-215`, localStorage). They never write `defaultModelSelection`, before or after. The composer effort path is the same before and after; at most it is a negative control.
4. **Healthy provider (enabled and available, saved model visible): the same before and after.** The before code already returned `saved` unchanged, so changing effort wrote the correct model. A recording of this case proves nothing about the fix.
5. **Not fixed (residual, same before and after):** the saved model is **hidden** via `providerModelPreferences[<id>].hiddenModels` on a healthy provider. `resolveEffectiveDefaultModelSelection` shows a different model on the same provider, and changing effort persists that substitute model. The PR test "still follows a hidden model on the same provider" pins this as intended, but it is the same class of silent overwrite as the reported bug. Flag it to the PR author or maintainers.

## Environment / persistence

- Dev server in the PR worktree with isolated state: `vp run dev` (state in `<wt>/.t3`). Find the file with `find /home/ubuntu/wt/pr-22193/.t3 -name settings.json`. It is expected at `.t3/userdata/settings.json`.
- **Edit settings.json only while the dev server is stopped**, then start it again. Stop the server by its captured PID (AGENTS rule 1).
- Verify what is persisted with `jq '.defaultModelSelection, .providerInstances' <settings.json>` before and after each step.
- Before/after: capture one run on `3b95e7831b` (detached checkout or second worktree) and one on `1ce968ca29`, from the **same seeded settings.json**.
- Effort ids from the bundled manifest (`apps/server/src/provider/model-manifest.json`):
  - Claude opus-4-8 / opus-5-5 / haiku-5-5: low, medium, high, xhigh, max (+ ultrathink/ultracode).
  - opus-4-5: low, medium, high, max (no xhigh).
  - sonnet-4-6: low, medium, high, max (no xhigh).
  - **haiku-4-5: no effort descriptor.**
  - Codex models use `reasoningEffort`.

### Seeds (merge into settings.json)

```jsonc
// S-base: saved Claude default with high effort
"defaultModelSelection": { "instanceId": "claudeAgent", "model": "claude-opus-4-8",
                           "options": [{ "id": "effort", "value": "high" }] }

// S-off: S-base + Claude turned off (same effect as the Providers page toggle)
"providerInstances": { "claudeAgent": { "driver": "claudeAgent", "enabled": false } }

// S-unavail: S-base + a config that fails decoding -> snapshot availability:"unavailable"
//   (ProviderInstanceRegistry.ts:100-118). Any config that fails ClaudeSettings decode works;
//   confirm on Settings > Providers that Claude shows an "Invalid config" error.
"providerInstances": { "claudeAgent": { "driver": "claudeAgent", "config": { "binaryPath": 123 } } }

// S-absent: saved instance id that is not in the snapshot at all
"defaultModelSelection": { "instanceId": "claude_work", "model": "claude-opus-4-8",
                           "options": [{ "id": "effort", "value": "high" }] }
```

Codex must stay enabled and ready (the default) so that a substitute exists. **Without a ready alternative provider, the before build shows "No providers available" rather than a substitute.**

## Matrix

| id | kind | seed | steps | expected BEFORE (3b95e7) | expected AFTER (1ce968) | persisted check |
|---|---|---|---|---|---|---|
| R1a | recording | S-off | Settings → General → New threads → Model. Open the effort/traits picker if shown and choose a different effort (e.g. Low). Reload. | Row shows the **Codex default model** with a Codex effort picker and no "turned off" note. Picking Low overwrites the saved model. After reload the row still shows Codex. | Row shows **Claude Opus 4.8** (or the raw slug), status note **"Claude is turned off."**, **no effort picker**, so effort cannot be changed here. After reload the row is unchanged. | Before: `.defaultModelSelection` becomes `{instanceId:"codex", model:<codex default>, options:[{id:"reasoningEffort",value:"low"}]}`. After: stays exactly S-base. |
| R1b | recording | S-unavail | Same as R1a | Same substitution and overwrite as R1a | Saved Claude model shown, note **"… is unavailable."** (check the display name text), no effort picker | Same as R1a |
| R1c | recording | S-absent | Same as R1a | Codex substitute with a Codex effort picker. Changing effort overwrites with codex. | Picker shows `claude-opus-4-8` (raw slug, no catalog entry), note **"This provider is not available on this environment."**, no effort picker | Before overwritten. After: `instanceId:"claude_work"` unchanged. |
| R1d | recording (control) | S-base, all healthy | Settings: change effort high→low, reload | Row shows Opus 4.8. Effort persists as low. Model unchanged. | **Same as before** | Both: `model:"claude-opus-4-8"`, effort `low`. Same before and after; this only shows no regression. |
| R1e | recording (control) | S-off or S-base | Open a draft thread in the composer, change effort in the composer traits menu, reload, open Settings | Composer effort changes the draft only | **Same** | `defaultModelSelection` unchanged in both builds. The composer never writes it. |
| R1f | recording (residual gap) | S-base + `"providerModelPreferences":{"claudeAgent":{"hiddenModels":["claude-opus-4-8"],"modelOrder":[]}}` | Settings: change effort, reload | Row shows a different Claude model (first visible). Changing effort persists that model. | **Same: the overwrite still happens** | Both: `model` changes from `claude-opus-4-8` to the substitute. Report as not covered by the fix. |
| R2a | recording | S-base with model `claude-opus-4-5` and `options:[{id:"effort",value:"xhigh"}]` (xhigh not offered) | Settings: open the effort picker without choosing, reload. Then choose `max`. | TraitsPicker shows a fallback or default label for the unsupported value. Nothing is written until the user picks. Picking writes opus-4-5 with max. | **Same.** TraitsPicker is untouched by the diff. | No write before an explicit pick. Afterwards `model` stays `claude-opus-4-5`. |
| R2b | recording | R2a seed + S-off | Settings Model row | Codex substitute (as in R1a) | Saved opus-4-5 shown, "Claude is turned off.", no effort picker | As in R1a |
| R3a | recording | `defaultModelSelection:{instanceId:"claudeAgent",model:"claude-haiku-4-5"}`, healthy | Settings Model row; confirm there is no effort control; change the model to opus-4-8 and back; reload | No effort control (or no effort entry in traits). Model persists. | **Same** | `model:"claude-haiku-4-5"`, no `options` written |
| R3b | still | R3a seed + S-off | Settings Model row | Codex substitute shown | Haiku 4.5 shown with "Claude is turned off." | — |
| P1 | still | S-off | Composer on a **new draft** with the default applied; also an existing thread whose modelSelection is Claude, with a hidden model, while Claude is off | Trigger may show the **first Claude option** instead of the saved model when the model is hidden or missing from the options | Trigger shows the saved model name or slug | Visual only |
| ST1–ST8 | stills | S-off (shows the change) and S-base (control) | Settings → General → New threads Model row, and the composer at rest; viewports **1440×900 and 390×844**, light and dark, before and after (2 surfaces × 2 widths × 2 themes × 2 builds = 16 shots per seed) | S-off: Codex model + Codex effort picker, no note | S-off: saved Claude model, status note, no effort picker. Check that the note does not overflow or wrap badly at 390px. | — |

## Unit tests

```
cd apps/web && vp test run src/components/chat/ProviderModelPicker.logic.test.ts \
  src/components/settings/ProjectDefaultsSettings.logic.test.ts
```
Ran at head: **2 files, 8 tests passed.** These test new functions only. Both functions are new files, so they cannot be run against the before build, and no test exercises the old overwrite path.

## Not provable at runtime / caveats

- The "settings turned the provider off before the snapshot updated" race (second test). It is timing-dependent, so assume the unit test covers it.
- Project-scope override (`projectSettingsOverrides[pid].providerInstances.claudeAgent.enabled=false`) goes through the same `entries`. It can be captured in project Settings scope, but it is not required.
- The display name in the "unavailable" note comes from the unavailable snapshot (`entry.displayName ?? driver`). It may read "claudeAgent is unavailable." Record the actual text.
- Whether a disabled provider's snapshot still carries `models`. If it does, the label shows "Opus 4.8"; if not, it shows the raw slug. Either is correct after the fix.
- The UX trade-off introduced: while the provider is off, the effort can no longer be edited in Settings at all. Reset and model change still work.
- Mobile has no equivalent of this Settings row in the diff. Desktop wraps web, so it is covered by the web captures.