{
  "scenario": "Homebrew stable cask ceiling gives a concrete route to Fable 5.2",
  "failsOnBase": true,
  "passesOnHead": true,
  "couldNotRun": null,
  "baseSha": "2770bf3c61c4d6b7bb1f5d3df74aacf54ac7a159",
  "headSha": "bf3e5643d784139efcfc2b6d3cdb0c8bd43d91d6",
  "verdict": "Proven: the scenario fails on base `2770bf3c61` and passes on head `bf3e5643d7`.",
  "steps": {
    "base": [
      {
        "step": "browser.emulateMedia colorScheme: dark",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /settings/providers?instanceId=claudeAgent",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(\"v2.1.311\").first()",
        "status": "passed"
      },
      {
        "step": "app.screenshot provider-settings",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /911d12f3-31a9-4fa7-a7f2-61fb022a90a5/mcp:b8aa839b-744b-4df9-9293-824ad83afb98",
        "status": "passed"
      },
      {
        "step": "locator.tap getByRole(\"button\", name: /Agent Homebrew claude-code cask latest/)",
        "status": "passed"
      },
      {
        "step": "locator.click locator(\"[data-chat-provider-model-picker-label]\").first()",
        "status": "passed"
      },
      {
        "step": "locator.fill getByPlaceholder(\"Search models...\")",
        "status": "passed"
      },
      {
        "step": "locator.waitFor getByRole(\"option\", name: /Claude Fable 5.1/) \u2192 visible",
        "status": "passed"
      },
      {
        "step": "browser.evaluate ",
        "status": "passed"
      },
      {
        "step": "app.screenshot model-picker",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(/the claude-code Homebrew cask only has v2.1.311/)",
        "status": "failed"
      }
    ],
    "head": [
      {
        "step": "browser.emulateMedia colorScheme: dark",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /settings/providers?instanceId=claudeAgent",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(\"v2.1.311\").first()",
        "status": "passed"
      },
      {
        "step": "app.screenshot provider-settings",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /f0130ed9-00a6-4ffe-a63b-1b26a22598dc/mcp:591a5ec7-0d43-4fe5-901d-dc2cc7fc9d25",
        "status": "passed"
      },
      {
        "step": "locator.tap getByRole(\"button\", name: /Agent Homebrew claude-code cask latest/)",
        "status": "passed"
      },
      {
        "step": "locator.click locator(\"[data-chat-provider-model-picker-label]\").first()",
        "status": "passed"
      },
      {
        "step": "locator.fill getByPlaceholder(\"Search models...\")",
        "status": "passed"
      },
      {
        "step": "locator.waitFor getByRole(\"option\", name: /Claude Fable 5.1/) \u2192 visible",
        "status": "passed"
      },
      {
        "step": "browser.evaluate ",
        "status": "passed"
      },
      {
        "step": "app.screenshot model-picker",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(/the claude-code Homebrew cask only has v2.1.311/)",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(/brew uninstall --cask claude-code && brew install --cask claude-code@latest/)",
        "status": "passed"
      }
    ]
  },
  "captures": [
    "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/001-provider-settings.png",
    "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/002-model-picker.png",
    "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/003-failure.png",
    "head/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/001-provider-settings.png",
    "head/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/002-model-picker.png"
  ],
  "videos": [
    "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/video/video.webm",
    "head/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/video/video.webm"
  ],
  "markdown": "### Scenario: Homebrew stable cask ceiling gives a concrete route to Fable 5.2\n\n**Proven: the scenario fails on base `2770bf3c61` and passes on head `bf3e5643d7`.**\n\n|  | Base `2770bf3c61` | Head `bf3e5643d7` |\n| --- | --- | --- |\n| Result | \u274c failed<br>ASSERTION_FAILED: expect.toBeVisible failed<br>locator: getByText(/the claude-code Homebrew cask only has v2.1.311/)<br>expected: visible<br>observed: no node (match count 0) | \u2705 passed |\n| provider settings | ![Base 2770bf3c61: provider settings](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/3ec10686ad2b7291/001-provider-settings.png) | ![Head bf3e5643d7: provider settings](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/fccc0961d06eb907/001-provider-settings.png) |\n| model picker | ![Base 2770bf3c61: model picker](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/f35c5a91c35a4305/002-model-picker.png) | ![Head bf3e5643d7: model picker](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/ea9cde4deac03369/002-model-picker.png) |\n| failure | ![Base 2770bf3c61: failure](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/1e1fe50ad573809e/003-failure.png) |  |\n",
  "base": {
    "sha": "2770bf3c61c4d6b7bb1f5d3df74aacf54ac7a159",
    "status": "failed",
    "title": "Homebrew stable cask ceiling gives a concrete route to Fable 5.2",
    "message": "ASSERTION_FAILED: expect.toBeVisible failed\nlocator: getByText(/the claude-code Homebrew cask only has v2.1.311/)\nexpected: visible\nobserved: no node (match count 0)",
    "steps": [
      {
        "step": "browser.emulateMedia colorScheme: dark",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /settings/providers?instanceId=claudeAgent",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(\"v2.1.311\").first()",
        "status": "passed"
      },
      {
        "step": "app.screenshot provider-settings",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /911d12f3-31a9-4fa7-a7f2-61fb022a90a5/mcp:b8aa839b-744b-4df9-9293-824ad83afb98",
        "status": "passed"
      },
      {
        "step": "locator.tap getByRole(\"button\", name: /Agent Homebrew claude-code cask latest/)",
        "status": "passed"
      },
      {
        "step": "locator.click locator(\"[data-chat-provider-model-picker-label]\").first()",
        "status": "passed"
      },
      {
        "step": "locator.fill getByPlaceholder(\"Search models...\")",
        "status": "passed"
      },
      {
        "step": "locator.waitFor getByRole(\"option\", name: /Claude Fable 5.1/) \u2192 visible",
        "status": "passed"
      },
      {
        "step": "browser.evaluate ",
        "status": "passed"
      },
      {
        "step": "app.screenshot model-picker",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(/the claude-code Homebrew cask only has v2.1.311/)",
        "status": "failed"
      }
    ],
    "captures": [
      "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/001-provider-settings.png",
      "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/002-model-picker.png",
      "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/003-failure.png"
    ],
    "videos": [
      "base/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/video/video.webm"
    ]
  },
  "head": {
    "sha": "bf3e5643d784139efcfc2b6d3cdb0c8bd43d91d6",
    "status": "passed",
    "title": "Homebrew stable cask ceiling gives a concrete route to Fable 5.2",
    "steps": [
      {
        "step": "browser.emulateMedia colorScheme: dark",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /settings/providers?instanceId=claudeAgent",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(\"v2.1.311\").first()",
        "status": "passed"
      },
      {
        "step": "app.screenshot provider-settings",
        "status": "passed"
      },
      {
        "step": "app.open <pairing link>",
        "status": "passed"
      },
      {
        "step": "expect.not.toHaveURL /\\/pair(?:[?#]|$)/",
        "status": "passed"
      },
      {
        "step": "app.open /f0130ed9-00a6-4ffe-a63b-1b26a22598dc/mcp:591a5ec7-0d43-4fe5-901d-dc2cc7fc9d25",
        "status": "passed"
      },
      {
        "step": "locator.tap getByRole(\"button\", name: /Agent Homebrew claude-code cask latest/)",
        "status": "passed"
      },
      {
        "step": "locator.click locator(\"[data-chat-provider-model-picker-label]\").first()",
        "status": "passed"
      },
      {
        "step": "locator.fill getByPlaceholder(\"Search models...\")",
        "status": "passed"
      },
      {
        "step": "locator.waitFor getByRole(\"option\", name: /Claude Fable 5.1/) \u2192 visible",
        "status": "passed"
      },
      {
        "step": "browser.evaluate ",
        "status": "passed"
      },
      {
        "step": "app.screenshot model-picker",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(/the claude-code Homebrew cask only has v2.1.311/)",
        "status": "passed"
      },
      {
        "step": "expect.toBeVisible getByText(/brew uninstall --cask claude-code && brew install --cask claude-code@latest/)",
        "status": "passed"
      }
    ],
    "captures": [
      "head/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/001-provider-settings.png",
      "head/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/screenshots/002-model-picker.png"
    ],
    "videos": [
      "head/results/homebrew-claude-ceiling-homebrew-stable-cask-ceiling-07daebaf9754a066/attempt-1/video/video.webm"
    ]
  }
}