
testability-scoring
AI-powered testability assessment using 10 principles of intrinsic testability with Playwright and optional Vibium integration. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports.
- Installs
- 114
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/proffesor-for-testing/agentic-qe --skill testability-scoring- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads testability-scoring automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Testability Scoring
Browser engine
Uses the qe-browser fleet skill (.claude/skills/qe-browser/) as the primary browser engine. Vibium is installed by aqe init. The legacy scripts/run-assessment.sh + Playwright path remains as a fallback when a team already has a Playwright test suite configured, but new runs should prefer:
vibium go "$TARGET_URL"
vibium wait load
vibium a11y-tree --json > /tmp/testability/tree.json
vibium eval --stdin --json <<'EOF' > /tmp/testability/signals.json
JSON.stringify({
headings: document.querySelectorAll('h1,h2,h3,h4,h5,h6').length,
testIds: document.querySelectorAll('[data-testid]').length,
forms: document.querySelectorAll('form').length,
ariaLabels: document.querySelectorAll('[aria-label]').length,
links: document.querySelectorAll('a[href]').length,
});
EOF
node .claude/skills/qe-browser/scripts/assert.js --checks '[
{"kind": "no_console_errors"},
{"kind": "no_failed_requests"}
]'
<default_to_action> When assessing testability:
- RUN assessment against target URL
- ANALYZE all 10 principles automatically
- GENERATE HTML report with radar chart
- PRIORITIZE improvements by impact/effort
- INTEGRATE with QX Partner for holistic view
Quick Assessment:
# Run assessment on any URL
TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
# Or use shell script wrapper
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/
The 10 Principles at a Glance:
| Principle | Weight | Key Question |
|---|---|---|
| Observability | 15% | Can we see what's happening? |
| Controllability | 15% | Can we control the application? |
| Algorithmic Simplicity | 10% | Are behaviors predictable? |
| Algorithmic Transparency | 10% | Can we understand what it does? |
| Algorithmic Stability | 10% | Does behavior remain consistent? |
| Explainability | 10% | Is the interface understandable? |
| Unbugginess | 10% | How error-free is it? |
| Smallness | 10% | Are components appropriately sized? |
| Decomposability | 5% | Can we test parts in isolation? |
| Similarity | 5% | Is the tech stack familiar? |
Grade Scale:
- A (90-100): Excellent testability
- B (80-89): Good testability
- C (70-79): Adequate testability
- D (60-69): Below average
- F (0-59): Poor testability </default_to_action>
Quick Reference Card
Running Assessments
| Method | Command | When to Use |
|---|---|---|
| Shell Script | ./scripts/run-assessment.sh URL | One-time assessment |
| ENV Override | TEST_URL='URL' npx playwright test... | CI/CD integration |
| Config File | Update tests/testability-scoring/config.js | Repeated runs |
Principle Details
High Weight (15% each)
| Principle | Measures | Indicators |
|---|---|---|
| Observability | State visibility, logging, monitoring | Console output, network tracking, error visibility |
| Controllability | Input control, state manipulation | API access, test data injection, determinism |
Medium Weight (10% each)
| Principle | Measures | Indicators |
|---|---|---|
| Simplicity | Predictable behavior | Clear I/O relationships, low complexity |
| Transparency | Understanding what system does | Visible processes, readable code |
| Stability | Consistent behavior | Change resilience, maintainability |
| Explainability | Interface understanding | Good docs, semantic structure, help text |
| Unbugginess | Error-free operation | Console errors, warnings, runtime issues |
| Smallness | Component size | Element count, script bloat, page complexity |
Low Weight (5% each)
| Principle | Measures | Indicators |
|---|---|---|
| Decomposability | Isolation testing | Component separation, modular design |
| Similarity | Technology familiarity | Standard frameworks, known patterns |
Assessment Workflow
1. Navigate to URL → 2. Collect Metrics → 3. Score Principles
↓
4. Generate JSON ← 5. Calculate Grades ← 6. Apply Weights
↓
7. Generate HTML Report with Radar Chart
↓
8. Open in Browser (auto-opens)
Output Files
tests/reports/
├── testability-results-<timestamp>.json # Raw data
├── testability-report-<timestamp>.html # Visual report
└── latest.json # Symlink
Integration Examples
CI/CD Integration
# GitHub Actions
- name: Testability Assessment
run: |
timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh ${{ env.APP_URL }}
- name: Upload Reports
uses: actions/upload-artifact@v3
with:
name: testability-reports
path: tests/reports/testability-*.html
QX Partner Integration
// Combine testability with QX analysis
const qxAnalysis = await Task("QX Analysis", {
target: 'https://example.com',
integrateTestability: true
}, "qx-partner");
// Returns combined insights:
// - QX Score: 78/100
// - Testability Integration: Observability 72/100
// - Combined Insight: Low observability may mask UX issues
Programmatic Usage
import { runTestabilityAssessment } from './testability';
const results = await runTestabilityAssessment('https://example.com');
console.log(`Overall: ${results.overallScore}/100 (${results.grade})`);
console.log('Recommendations:', results.recommendations);
Agent Integration
// Run testability assessment
const assessment = await Task("Testability Assessment", {
url: 'https://example.com',
generateReport: true,
openBrowser: true
}, "qe-quality-analyzer");
// Use with QX Partner for holistic analysis
const qxReport = await Task("Full QX Analysis", {
target: 'https://example.com',
integrateTestability: true,
detectOracleProblems: true
}, "qx-partner");
Vibium Integration (Optional)
Overview
Vibium browser automation can be used alongside Playwright for enhanced testability assessment. While Playwright remains the primary engine, Vibium offers complementary capabilities for certain metrics.
Installation:
claude mcp add vibium -- npx -y vibium
Vibium-Enhanced Metrics
| Principle | Vibium Enhancement | Benefit |
|---|---|---|
| Observability | Auto-wait duration tracking | Measures DOM stability (30s timeout, 100ms polling) |
| Controllability | Element interaction success rate | Validates automation readiness via MCP |
| Stability | Screenshot consistency | Visual regression detection for layout stability |
| Explainability | Element attribute extraction | ARIA labels, semantic HTML validation |
When to Use Vibium
✅ USE Vibium for:
- Element stability metrics (auto-wait duration analysis)
- Visual consistency checks (screenshot comparison)
- MCP-native AI agent integration
- Lightweight Docker images (400MB vs 1.2GB)
❌ USE Playwright for:
- Console error detection (Vibium V1 lacks console API)
- Network performance metrics (BiDi network APIs coming in V2)
- Comprehensive browser coverage (Firefox, Safari)
- Production-proven stability (Vibium V1 released Dec 2024)
Hybrid Assessment Example
// Testability assessment using both engines
const assessment = {
// Playwright: Comprehensive metrics
playwright: await runPlaywrightAssessment(url),
// Vibium: Stability metrics
vibium: {
elementStability: await measureAutoWaitDuration(url),
visualConsistency: await compareScreenshots(url),
accessibilityAttributes: await extractARIALabels(url)
}
};
// Enhanced Observability Score
const observability =
(assessment.playwright.consoleErrors * 0.6) +
(assessment.vibium.elementStability * 0.4);
Vibium MCP Tools for Testability
// 1. Element Stability Measurement
const browser = await browser_launch();
await browser_navigate({ url });
const startTime = Date.now();
const element = await browser_find({ selector: ".critical-element" });
const autoWaitDuration = Date.now() - startTime;
// Lower duration = better stability
// 2. Visual Consistency Check
const screenshot1 = await browser_screenshot();
await browser_navigate({ url }); // Reload
const screenshot2 = await browser_screenshot();
const visualDiff = compareImages(screenshot1.png, screenshot2.png);
// Lower diff = better stability
// 3. Accessibility Attribute Extraction
const elements = await browser_find({ selector: "button, a, input" });
const ariaLabels = elements.map(el => el.attributes["aria-label"]);
const semanticScore = (ariaLabels.filter(Boolean).length / elements.length) * 100;
Migration Strategy
Current (V2.2): Hybrid approach
- Playwright: Primary engine for all 10 principles
- Vibium: Optional enhancement for stability metrics
Future (V3.0): When Vibium V2 ships
- Evaluate Vibium as primary engine if:
- Console/Network APIs available
- Production stability proven
- Community adoption increases
Agent Coordination Hints
Memory Namespace
aqe/testability/
├── assessments/* - Assessment results by URL
├── historical/* - Historical scores for trend analysis
├── recommendations/* - Improvement recommendations
├── integration/* - QX integration data
└── vibium/* - Vibium-specific metrics (optional)
Fleet Coordination
const testabilityFleet = await FleetManager.coordinate({
strategy: 'testability-assessment',
agents: [
'qe-quality-analyzer', // Primary assessment
'qx-partner', // UX integration
'qe-visual-tester' // Visual validation
],
topology: 'sequential'
});
Common Issues & Solutions
| Issue | Solution |
|---|---|
| Tests timing out | Increase timeout: timeout 300 ./scripts/run-assessment.sh URL |
| Partial results | Check console errors, increase network timeout |
| Report not opening | Use AUTO_OPEN=false, open manually |
| Config not updating | Use TEST_URL env var instead |
| Vibium not available | Install via claude mcp add vibium -- npx -y vibium (optional) |
| Hybrid mode errors | Vibium is optional; assessments work without it |
Related Skills
- accessibility-testing - WCAG compliance (overlaps with Explainability)
- visual-testing-advanced - UI consistency
- performance-testing - Load time metrics
Credits & References
Framework Origin
- Heuristics for Software Testability by James Bach and Michael Bolton
- Available at: https://www.satisfice.com/download/heuristics-of-software-testability
Implementation
- Based on https://github.com/fndlalit/testability-scorer (contributed by @fndlalit)
- Playwright v1.49.0+ with AI capabilities (primary engine)
- Vibium v1.0+ with MCP integration (optional enhancement)
- Chart.js for radar visualizations
Vibium Resources
- GitHub: https://github.com/VibiumDev/vibium
- MCP Integration:
claude mcp add vibium -- npx -y vibium - Created by Jason Huggins (creator of Selenium/Appium)
Remember
Testability is an investment, not an afterthought.
Good testability:
- Reduces debugging time
- Enables faster feedback loops
- Makes defects easier to find
- Supports continuous testing
Low scores = High risk. Prioritize improvements by weight × impact.
FAQ
What does the testability-scoring skill do?
AI-powered testability assessment using 10 principles of intrinsic testability with Playwright and optional Vibium integration. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports.
How do I install testability-scoring?
Run npx skills add https://github.com/proffesor-for-testing/agentic-qe --skill testability-scoring in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/testability-scoring/ (personal) or .claude/skills/testability-scoring/ (project).
Which AI agents can use testability-scoring?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is testability-scoring safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains testability-scoring?
It is published by proffesor-for-testing in proffesor-for-testing/agentic-qe. It has 114 recorded installs.
More from proffesor-for-testing/agentic-qe
code-review-quality
proffesor-for-testing/agentic-qe
Conduct context-driven code reviews focusing on quality, testability, and maintainability. Use when reviewing code, providing feedback, or establishing review practices.
compatibility-testing
proffesor-for-testing/agentic-qe
Cross-browser, cross-platform, and cross-device compatibility testing ensuring consistent experience across environments. Use when validating browser support, testing responsive design, or ensuring platform compatibility.
api-testing-patterns
proffesor-for-testing/agentic-qe
Comprehensive API testing patterns including contract testing, REST/GraphQL testing, and integration testing. Use when testing APIs or designing API test strategies.
regression-testing
proffesor-for-testing/agentic-qe
Strategic regression testing with test selection, impact analysis, and continuous regression management. Use when verifying fixes don't break existing functionality, planning regression suites, or optimizing test execution for faster feedback.
test-automation-strategy
proffesor-for-testing/agentic-qe
Design and implement effective test automation with proper pyramid, patterns, and CI/CD integration. Use when building automation frameworks or improving test efficiency.
github-code-review
proffesor-for-testing/agentic-qe
Comprehensive GitHub code review with AI-powered swarm coordination
Popular testing & qa skills
tdd
mattpocock/skills
Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
grilling
mattpocock/skills
Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
migrate-to-shoehorn
mattpocock/skills
Migrate test files from as type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace as in tests, or needs partial test data.
run-train
lllllllama/rigorpilot-skills
Rigor Train skill for deep learning research repositories. Use when a documented or selected training command should be run conservatively for startup verification, short-run verification, full kickoff, or resume, with command, config, seed, log, checkpoint, status, and metric evidence written to standardized trainoutputs/. Do not use for environment setup, exploratory sweeps, speculative idea implementation, or end-to-end orchestration.
test-driven-development
obra/superpowers
Use when implementing any feature or bugfix, before writing implementation code
finishing-a-development-branch
obra/superpowers
Use when implementation is complete, all tests pass, and you need to decide how to integrate the work