Browser Max Automation

Browser automation via Playwright MCP, existing-browser CDP, and direct CDP helpers.

When to Use

• ブラウザ自動化、UI 確認、フォーム操作、スクリーンショット取得
• MCP で手順を確立してから Python で一括実行したいとき
• 既存ブラウザのログイン状態を CDP 経由で使いたいとき
• Playwright MCP / connect_over_cdp() が不安定で、raw CDP WebSocket に切り替えたいとき
• モーダルや file chooser など、通常クリックが壊れやすい UI を扱うとき

Not the Best Fit

• API / CLI を先に検討: 公式 API（YouTube Data API、Microsoft Graph、GitHub API 等）や CLI で read/write が完結するならそちらを優先し、UI が脆い兆候（native file chooser / shadow DOM / 多段ウィザード / レンダラーを止める modal）で固執せず切り替える（例: YouTube Studio UI → captions.insert）
• URL の生存確認は HTTP client で行うが、dead 判定だけはブラウザまでエスカレートする。確実な消失の証拠は HTTP 404 / 410 だけ。403 は bot ブロック、timeout は一過性、SSL の unable to get local issuer certificate は中間 CA を返さないサーバーで起きる（ブラウザと .NET/schannel は AIA で自動補完するが Python ssl はしない）。Invoke-WebRequest -Uri <url> -Method Get またはブラウザで裏取りしてから結論を出す
• PowerPoint / Loop / Dynamics 365 の expense entry 画面など専用 skill がある UI は、該当 skill の操作ルールを優先する
• 認証情報、秘密情報、MFA 応答をチャットで受け取らない。必要な入力はブラウザ上でユーザーに処理してもらう。ユーザー操作の後は、リカバリーコード・token・password を表示し得る画面の本文を読まず URL（/json/list）だけで状態を確かめる。「完了」の申告後も確定前の秘密表示画面が残っていることがある
• 例外: 対象サイトが passkey / FIDO2 / WebAuthn 対応なら、CDP 仮想認証機で完全無人化できる。優先順位は passkey (仮想認証機) > メール OTP > SMS > 物理キー / 生体。詳細は references/instructions/webauthn-virtual-authenticator.md を参照する

Choose Mode First

• Default to a visible, headed browser. Use headless only when explicitly requested, never as an automatic recovery path. Reuse a verified authenticated session and an owned work tab when possible; a new profile does not imply reusable authentication. When a dedicated launch is already authorized, an absent session calls for an owned launch, not an unavailable verdict.
• Preserve the OS foreground window and the user's selected tab. Do not routinely call bring_to_front, Page.bringToFront, Target.activateTarget, or native focus/keystroke helpers. This applies to browser launch, tab creation, navigation, capture, and recovery, not just clicks.
• If a necessary operation cannot avoid activation, explain why and which bounded step needs it before proceeding. Do not force focus back afterward: the user may have switched applications meanwhile. Credentials and MFA remain user-entered. When handing a step to the user, give the window title and confirm the window is on the primary monitor (multi-monitor setups hide it otherwise).
• Identify the work tab, goal, and authorized side effects. Browser access or drafting permission does not authorize sending messages, purchases, trades, cancellations, or contract changes; bind approval to the destination and exact content/action. Read-only helpers must not submit or select settings options; menu expansion is allowed, but choosing an item may auto-save. Report attempted writes, including blocked ones. Pause on user editing, target drift, or lost ownership; viewing a tab does not permit discarding it.
• Prefer API/CLI helpers for supported data operations, but preserve UI execution when the goal is a demo, UI verification, or observation of the browser workflow.

既存ブラウザ CDP の起動、profile 確認、port drift、認証 URL encode、スクリーンショット取得は references/instructions/cdp-existing-browser.md を参照する。 直接 WebSocket CDP の起動フラグ、websocket-client 接続、SPA hash navigation、virtual scroll 操作は references/instructions/cdp-direct-websocket.instructions.md を参照する。

Core Loop

1. Reuse a suitable script or establish the flow on an owned, verified work tab.
2. Resolve the target and expected postcondition with a scoped snapshot or query.
3. Perform the action once; wait for the expected state with a deadline.
4. Read back the durable result before retrying or moving to the next item.

Keep one working control route instead of repeatedly switching MCP/CLI/CDP. Batch independent reads and return compact results; use full snapshots or screenshots at meaningful visual checkpoints, not after every read.

For downloads, save a durable extension-bearing copy before context/runner cleanup and verify its bytes/content, not its browser-history name or icon; see download handling.

Wait for required controls or an explicit empty state. A title, route or tab-click success proves neither authentication nor that the selected view matches rendered rows. Distinguish parse failure from empty data: preserve raw text, normalize observed invisible label characters only in an extraction copy, and test explicit zero, missing fields and mismatched IDs through the collector. Never inject an expected ID to pass validation. A readiness deadline ends in unverified state, not inferred logout.

UI verification では、操作前に期待する state と確認方法を決める。成功 toast やボタン押下だけを成功判定にせず、DOM、URL、永続化された一覧行、API の read 結果、または screenshot / trace などの証跡で確認する。 For settings readback, locate the semantic section/group and inspect actual roles before choosing a locator: labels may move from radios to dropdowns, and similar controls may govern unrelated scopes. Read selection from checked, aria-checked, or the displayed value. A current selection or documented default does not establish the creation-time value; require historical and unchanged-state evidence, otherwise report it unverified. For replies/messages, exclude the composer and list previews from success checks. Capture POST status and reply/message ID when available; otherwise verify the approved body and sender appear once in the intended thread after a fresh read/reload. Preserve drafts before reload, never discard another person's input, and report unavailable network evidence as unobserved. Stale counts or a retained draft after submission are not grounds to resend.

API と DOM の状態が一時的にずれるケースがある。API が stale / capture failure を返しても、画面上の cell / row の aria-label や status text が進展した状態を示しているなら DOM も正本候補として扱う。API 単独で「未完了」と断定しない。

Decision Patterns

CLI commands alone do not remove per-action reasoning. Save a verified sequence with stable locators, input/target parameters, readback, duplicate-write protection, and resumable per-item results. Store task-specific scripts with their project and generic helpers with the owning skill; record purpose and invocation for discovery. Do not persist secrets or transient snapshot refs. See repeatable workflows.

Virtualized feeds / per-item mutations and handler-count diagnosis: see UI Fallbacks.

unsaved editor / draft タブを壊さない

Keep dirty editor tabs in place; use a separate clean work tab instead of navigating or reloading them. Never auto-accept Leave or discard changes. CDP reporting No dialog is showing does not rule out a browser-chrome prompt. Native automation may invoke Cancel/Stay only when the owned browser process, selected-tab address, exact leave-site warning and enabled button uniquely match; otherwise stop. Never handle credential, permission or arbitrary dialogs this way. After dismissal, verify the same target responds; preserved form values are not proof of saved settings. See unsaved-form safety and native recovery.

iframe、force click、file chooser、hidden input、evaluate+fetchは UI Fallbacks を参照する。force: true は要素の実在と可視性を確認した後の最終手段とする。

Safety Rules

技術・セキュリティ内容を外部プラットフォームへ繰り返し／一括送信する前の content-filter preflight は UI Fallbacks を参照する。

Failure Budget

• Before a write, choose its durable success signal and one bounded recovery. Allow the initial attempt plus at most one recovery attempt per logical operation across UI, CLI, CDP, script, and replacement-tab routes; switching routes does not reset the budget.
• A timeout is not proof of failure. Re-read authoritative state; retry only when non-application is established or an idempotency mechanism prevents duplication. If the outcome remains ambiguous, stop without replaying the write.
• Stop after two failed attempts and report target, attempts, last state, and restart condition. Waiting for normal asynchronous progress is not another attempt: use a deadline and a narrow state check, not fixed sleeps or repeated full-page snapshots.
• UI clickからDOM mutation、keyboard、private frontend internalsへ連続的にfallbackしない。公開されていないViewModelやupload実装の操作は、明示承認のある単発incident以外では使わない
• Session ended、login redirect、別contextを検出したらstale DOMを操作しない。再認証後は永続stateを再取得し、未完了操作だけを再開する

CDP 排他制御

• Use one controller per work target. If another session owns the browser and coordination is unverified, use both a free port and a separate unused user-data-dir. A different port or profile-directory alone does not isolate a Chromium process; do not attach, restart it or copy its credentials to bypass ownership.
• Pin the verified profile/context and owned target ID; re-check URL, route, query, and item ID before each write batch. Never select the first domain match again after binding the work target.
• Before a necessary handoff, stop the owned runner and verify the current tool's detach/close semantics. Do not use browser_close as a generic disconnect: it may destroy tabs or the browser. Preserve dirty tabs; if safe detachment is unavailable, retain the working route or stop.
• Close only explicitly owned, no-longer-needed targets after result verification. Never close the last tab of a shared browser or stop another session's runner as cleanup. When Playwright MCP is attached over CDP, browser_close may only detach and leave the owned tab open; re-list targets and close the owned one by ID (/json/close/<id>).

raw WebSocket を使う場合は、CDP command id で応答をフィルタし、Runtime.enable / Page.enable など必要な domain を先に有効化する。詳細は references/instructions/cdp-direct-websocket.instructions.md を参照する。

Playwright MCP の file: 拒否（http.server で配信）と破綻しやすい場面の一覧は CDP Recovery and Context を参照する。

Reference Map

Done Criteria

• Verify the actual control route preserves OS foreground and user tab selection during the operation, not just before/after it; distinguish measured results from unverified behavior. Inspect saved captures for incomplete rendering.
• Report the work target and durable result; for repeat runs, report script reuse and compare elapsed time, tool calls, and retries without omitting verification. See the acceptance scenarios.
• 成功判定を toast だけに頼らず、DOM / URL / API read / screenshot / status artifact のいずれかで確認している
• 失敗時は modal / file chooser / iframe / CDP context のどこで詰まったかを報告している
• Playwright MCP が作業 repo 直下へ書く .playwright-mcp/（console log / snapshot）が .gitignore 済みで、自分が作った未追跡分は git clean -n で確認して削除している。追跡済みなら git rm -r --cached -- .playwright-mcp（ファイルは残る）