vitest-midscene-e2e

Q: How do I install vitest-midscene-e2e?

Run npx skills add https://github.com/web-infra-dev/midscene-skills --skill vitest-midscene-e2e in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/vitest-midscene-e2e/ (personal) or .claude/skills/vitest-midscene-e2e/ (project).

by web-infra-devTesting & QA

Enhances Vitest with Midscene for AI-powered UI testing across Web (Playwright), Android (ADB), and iOS (WDA). Scaffolds new projects, converts existing projects, and creates/updates/debugs/runs E2E tests using natural-language UI interactions. Triggers: write test, add test, create test, update test, fix test, debug test, run test, e2e test, midscene test, new project, convert project, init project, 写测试, 加测试, 创建测试, 更新测试, 修复测试, 调试测试, 运行测试, 新建工程, 转化工程.

Installs: 2.1K
Last 24h: —
Trending rank: —
Security: Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/web-infra-dev/midscene-skills --skill vitest-midscene-e2e

01
Install
Run the command in your project. The CLI asks which agents to install for.
02
Describe the task
Ask your agent for something this skill covers. It loads vitest-midscene-e2e automatically when relevant.
03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

Vitest Midscene E2E

Modules

Module	Role
Vitest	TypeScript test framework. Provides `describe`/`it`/`expect`/hooks for test organization, assertions, and lifecycle.
Midscene	AI-driven UI automation. Interacts with UI elements via natural language — no fragile selectors. Core API: `aiAct`.

Supported platforms:

Web — WebTest (Playwright Chromium): ctx.agent + ctx.page
Android — AndroidTest (ADB + scrcpy): ctx.agent only
iOS — IOSTest (WebDriverAgent): ctx.agent only

Workflow

Step 1: Clone boilerplate & ensure project ready

bash scripts/clone-boilerplate.sh

The boilerplate at ~/.midscene/boilerplate/vitest-all-platforms-demo/ is the canonical reference for project structure, configs, platform context classes, and test conventions. Compare the current project against it. If anything is missing, ask the user which platform(s) they need (Web / Android / iOS), then fill in what's missing using the boilerplate as the target state. Only include files for the requested platform(s). Do NOT overwrite existing configs or files. Copy .env.example from the boilerplate as .env if it doesn't exist, and prompt the user to fill in the env vars.

Step 2: Read the Midscene Agent API section below before writing tests

It contains mandatory rules for using aiAct — the primary API for all UI operations. Do NOT skip this step.

Step 3: Create, update, or run tests

Use the boilerplate's e2e/ directory and src/context/ as reference for patterns and conventions. Before running tests, ensure dependencies are installed and .env is configured. When debugging failures, check troubleshooting.md.

Midscene Agent API

ctx.agent is a platform-specific agent instance. All methods return Promises.

Web: PlaywrightAgent from @midscene/web/playwright
Android: AndroidAgent from @midscene/android
iOS: IOSAgent from @midscene/ios

All three agents share the same AI methods below.

Mandatory Rule: Use `aiAct` for User-Described Steps

When the user describes a UI action or state confirmation in natural language, you MUST use aiAct to implement it. Do NOT decompose user instructions into aiTap/aiInput/aiAssert or other fine-grained APIs. Pass the user's intent directly to aiAct and let Midscene's AI handle the planning and execution.

// User says: "type iPhone in the search box and click search"

// WRONG — manually decomposing into fine-grained APIs
await ctx.agent.aiInput('search box', { value: 'iPhone' });
await ctx.agent.aiTap('search button');

// CORRECT — pass intent directly to aiAct
await ctx.agent.aiAct('type "iPhone" in the search box, then click the search button');

Assertions, data extraction, and waiting should also be done via aiAct — it handles all of these. Do NOT use aiAssert, aiQuery, aiWaitFor, aiTap, or aiInput separately.

aiAct(taskPrompt, opt?) — Primary API

aiAct is the primary API for all UI operations and state confirmations. It accepts natural language instructions and autonomously plans and executes multi-step interactions.

// UI operations
await ctx.agent.aiAct('type "iPhone" in the search box, then click the search button');
await ctx.agent.aiAct('hover over the user avatar in the top right');

// State confirmations / assertions — also use aiAct
await ctx.agent.aiAct('verify the page shows "Login successful"');
await ctx.agent.aiAct('verify the error message is visible');

Prompt-driven File Uploads (Web only)

When an aiAct prompt asks Midscene to upload files, pass fileChooserAllowedDir explicitly. Use the smallest directory containing that test case's fixtures, and refer to files relative to it in the prompt. Do not use the project root or a home directory. Replace ./fixtures below with the fixture directory relative to the test process working directory.

await ctx.agent.aiAct(
  'click the upload button and upload avatar.png',
  { fileChooserAllowedDir: './fixtures' },
);

Phase splitting: If the task prompt is too long or covers multiple distinct stages, split it into separate aiAct calls — one per phase. Each phase should be a self-contained logical step, and all phases combined must match the user's original intent.

// Incorrect — prompt spans multiple pages and too many steps, AI may lose context mid-way
await ctx.agent.aiAct('click the settings button in the top nav, go to settings page, find personal info and click into it, change email to "test@example.com", change phone to "13800000000", click save, wait for success');

// Correct — split by page/stage boundary, each phase stays within one logical context
await ctx.agent.aiAct('click the settings button in the top nav, go to settings page, find personal info and click into it');
await ctx.agent.aiAct('change email to "test@example.com", change phone to "13800000000", click save');
await ctx.agent.aiAct('verify the save success message appears');

aiAction is deprecated. Use aiAct or ai instead.

Common Mistakes

Vague locators — 'button' is ambiguous; use 'the blue "Submit" button at the top of the page'
Deprecated aiAction — use aiAct instead
Ambiguous multi-element targets — specify row/position: 'the delete button in the first product row'

Agent Configuration — `aiActionContext`

aiActionContext is a system prompt string appended to all AI actions performed by the agent. Use it to define the AI's role and expertise.

// Set via agentOptions in setup()
const ctx = WebTest.setup('https://example.com', {
  agentOptions: {
    aiActionContext: 'You are a Web UI testing expert.',
  },
});

Good examples:

'You are a Web UI testing expert.'
'You are an Android app testing expert who is familiar with Chinese UI.'

Bad examples:

'Click the login button.' — specific actions belong in aiAct(), not aiActionContext
'The page is in Chinese.' — this is page description, not a system prompt

How to Look Up More

In node_modules/@midscene/web, node_modules/@midscene/android, and node_modules/@midscene/ios, find the type definitions for the agent classes
If types are not enough, follow the source references in the .d.ts files to read the implementation code in node_modules
Download https://midscenejs.com/llms.txt, then use grep to search for the API or concept you need (the file is large, do not read it in full)

FAQ

What does the vitest-midscene-e2e skill do?

How do I install vitest-midscene-e2e?

Run npx skills add https://github.com/web-infra-dev/midscene-skills --skill vitest-midscene-e2e in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/vitest-midscene-e2e/ (personal) or .claude/skills/vitest-midscene-e2e/ (project).

Which AI agents can use vitest-midscene-e2e?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is vitest-midscene-e2e safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains vitest-midscene-e2e?

It is published by web-infra-dev in web-infra-dev/midscene-skills. It has 2,073 recorded installs.

More from web-infra-dev/midscene-skills

View all →

browser-automation

web-infra-dev/midscene-skills

Vision-driven browser automation using Midscene. Operates from screenshots — no DOM or accessibility labels needed. Runs in headless Puppeteer — does NOT take over the user's mouse or keyboard. Also supports CDP mode and Bridge mode to connect to an existing Chrome. Use this skill when the user wants to: - Browse, navigate, or open web pages - Scrape, extract, or collect data from websites - Fill out forms, upload local files, click buttons, or interact with web elements - Verify, validate, test, or QA frontend UI behavior - Take screenshots of web pages - Automate multi-step web workflows - Test what was just built, see if it works in browser - Connect to Chrome via CDP, DevTools Protocol, or remote debugging - Connect to user's Chrome browser, control my browser, operate my Chrome Powered by Midscene.js (https://midscenejs.com)

Productivity & Automation5.8K installs

android-device-automation

web-infra-dev/midscene-skills

Vision-driven Android device automation using Midscene. Operates entirely from screenshots — no DOM or accessibility labels required. Can interact with all visible elements on screen regardless of technology stack. Control Android devices with natural language commands via ADB. Perform taps, swipes, text input, app launches, screenshots, and more. Trigger keywords: android, phone, mobile app, tap, swipe, install app, open app on phone, android device, mobile automation, adb, launch app, mobile screen, test android app, verify mobile app, QA on phone, check the app on android, test on device, see if the app works on phone, end-to-end test on android, visual verification on mobile Powered by Midscene.js (https://midscenejs.com)

Mobile5.8K installs

ios-device-automation

web-infra-dev/midscene-skills

Vision-driven iOS device automation using Midscene CLI. Operates entirely from screenshots — no DOM or accessibility labels required. Can interact with all visible elements on screen regardless of technology stack. Control iOS devices with natural language commands via WebDriverAgent. Triggers: ios, iphone, ipad, ios app, tap on iphone, swipe, mobile app ios, ios device, ios testing, iphone automation, ipad automation, ios screen, ios navigate, test ios app, verify on iphone, QA on ipad, check the app on ios, test on ios device, see if the app works on iphone, end-to-end test on ios, visual verification on ios Powered by Midscene.js (https://midscenejs.com)

Mobile3.3K installs

desktop-computer-automation

web-infra-dev/midscene-skills

Vision-driven desktop automation using Midscene. Control your local desktop (macOS, Windows, Linux) or a remote Windows desktop over RDP with natural language commands. Operates entirely from screenshots — no DOM or accessibility labels required. Can interact with all visible elements on screen regardless of technology stack. ⚠️ In local mode this takes over the user's real mouse and keyboard. For web apps, prefer "Browser Automation" instead. Only use this for desktop-native apps (Electron, Qt, native macOS/Windows/Linux) that cannot run in a browser, or for driving a remote Windows host via RDP. Triggers: open app, press key, desktop, computer, click on screen, type text, screenshot desktop, launch application, switch window, desktop automation, control computer, mouse click, keyboard shortcut, screen capture, find on screen, read screen, verify window, close app, test Electron app, rdp, remote desktop, windows server, connect via rdp Powered by Midscene.js (https://midscenejs.com)

Productivity & Automation3.3K installs

harmonyos-device-automation

web-infra-dev/midscene-skills

Vision-driven HarmonyOS NEXT device automation using Midscene. Operates entirely from screenshots — no DOM or accessibility labels required. Can interact with all visible elements on screen regardless of technology stack. Control HarmonyOS devices with natural language commands via HDC. Perform taps, swipes, text input, app launches, screenshots, and more. Trigger keywords: harmony, harmonyos, 鸿蒙, hdc, huawei device, harmony app, harmony automation, harmony phone, harmony tablet, test harmony app, verify on harmonyos, QA on 鸿蒙, check the app on harmony, test on huawei device, see if the app works on harmony, end-to-end test on harmonyos, visual verification on 鸿蒙 Powered by Midscene.js (https://midscenejs.com)

Productivity & Automation2.3K installs

computer-automation

web-infra-dev/midscene-skills

Productivity & Automation2.2K installs

Popular testing & qa skills

View category →

tdd

mattpocock/skills

Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.

Testing & QA1.2M installs

grilling

mattpocock/skills

Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.

Testing & QA996.2K installs

migrate-to-shoehorn

mattpocock/skills

Migrate test files from as type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace as in tests, or needs partial test data.

Testing & QA484.1K installs

run-train

lllllllama/rigorpilot-skills

Rigor Train skill for deep learning research repositories. Use when a documented or selected training command should be run conservatively for startup verification, short-run verification, full kickoff, or resume, with command, config, seed, log, checkpoint, status, and metric evidence written to standardized trainoutputs/. Do not use for environment setup, exploratory sweeps, speculative idea implementation, or end-to-end orchestration.

Testing & QA337.8K installs

test-driven-development

obra/superpowers

Use when implementing any feature or bugfix, before writing implementation code

Testing & QA261.8K installs

finishing-a-development-branch

obra/superpowers

Use when implementation is complete, all tests pass, and you need to decide how to integrate the work

Testing & QA216.8K installs

vitest-midscene-e2e

Install

Install

Describe the task

Review the result

SKILL.md

Vitest Midscene E2E

Modules

Workflow

Step 1: Clone boilerplate & ensure project ready

Step 2: Read the Midscene Agent API section below before writing tests

Step 3: Create, update, or run tests

Midscene Agent API

Mandatory Rule: Use aiAct for User-Described Steps

aiAct(taskPrompt, opt?) — Primary API

Prompt-driven File Uploads (Web only)

Common Mistakes

Agent Configuration — aiActionContext

How to Look Up More

FAQ

More from web-infra-dev/midscene-skills

Popular testing & qa skills

Mandatory Rule: Use `aiAct` for User-Described Steps

Agent Configuration — `aiActionContext`