MCPInstall

testability-scoring

AI-powered testability assessment using 10 principles of intrinsic testability with Playwright and optional Vibium integration. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports.

Installs
114
Last 24h
—
Trending rank
—
Security
Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/proffesor-for-testing/agentic-qe --skill testability-scoring
  1. 01

    Install

    Run the command in your project. The CLI asks which agents to install for.

  2. 02

    Describe the task

    Ask your agent for something this skill covers. It loads testability-scoring automatically when relevant.

  3. 03

    Review the result

    The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

Testability Scoring

Browser engine

Uses the qe-browser fleet skill (.claude/skills/qe-browser/) as the primary browser engine. Vibium is installed by aqe init. The legacy scripts/run-assessment.sh + Playwright path remains as a fallback when a team already has a Playwright test suite configured, but new runs should prefer:

vibium go "$TARGET_URL"
vibium wait load
vibium a11y-tree --json > /tmp/testability/tree.json
vibium eval --stdin --json <<'EOF' > /tmp/testability/signals.json
JSON.stringify({
  headings: document.querySelectorAll('h1,h2,h3,h4,h5,h6').length,
  testIds: document.querySelectorAll('[data-testid]').length,
  forms: document.querySelectorAll('form').length,
  ariaLabels: document.querySelectorAll('[aria-label]').length,
  links: document.querySelectorAll('a[href]').length,
});
EOF
node .claude/skills/qe-browser/scripts/assert.js --checks '[
  {"kind": "no_console_errors"},
  {"kind": "no_failed_requests"}
]'

<default_to_action> When assessing testability:

  1. RUN assessment against target URL
  2. ANALYZE all 10 principles automatically
  3. GENERATE HTML report with radar chart
  4. PRIORITIZE improvements by impact/effort
  5. INTEGRATE with QX Partner for holistic view

Quick Assessment:

# Run assessment on any URL
TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1

# Or use shell script wrapper
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/

The 10 Principles at a Glance:

PrincipleWeightKey Question
Observability15%Can we see what's happening?
Controllability15%Can we control the application?
Algorithmic Simplicity10%Are behaviors predictable?
Algorithmic Transparency10%Can we understand what it does?
Algorithmic Stability10%Does behavior remain consistent?
Explainability10%Is the interface understandable?
Unbugginess10%How error-free is it?
Smallness10%Are components appropriately sized?
Decomposability5%Can we test parts in isolation?
Similarity5%Is the tech stack familiar?

Grade Scale:

  • A (90-100): Excellent testability
  • B (80-89): Good testability
  • C (70-79): Adequate testability
  • D (60-69): Below average
  • F (0-59): Poor testability </default_to_action>

Quick Reference Card

Running Assessments

MethodCommandWhen to Use
Shell Script./scripts/run-assessment.sh URLOne-time assessment
ENV OverrideTEST_URL='URL' npx playwright test...CI/CD integration
Config FileUpdate tests/testability-scoring/config.jsRepeated runs

Principle Details

High Weight (15% each)

PrincipleMeasuresIndicators
ObservabilityState visibility, logging, monitoringConsole output, network tracking, error visibility
ControllabilityInput control, state manipulationAPI access, test data injection, determinism

Medium Weight (10% each)

PrincipleMeasuresIndicators
SimplicityPredictable behaviorClear I/O relationships, low complexity
TransparencyUnderstanding what system doesVisible processes, readable code
StabilityConsistent behaviorChange resilience, maintainability
ExplainabilityInterface understandingGood docs, semantic structure, help text
UnbugginessError-free operationConsole errors, warnings, runtime issues
SmallnessComponent sizeElement count, script bloat, page complexity

Low Weight (5% each)

PrincipleMeasuresIndicators
DecomposabilityIsolation testingComponent separation, modular design
SimilarityTechnology familiarityStandard frameworks, known patterns

Assessment Workflow

1. Navigate to URL → 2. Collect Metrics → 3. Score Principles
                                              ↓
4. Generate JSON ← 5. Calculate Grades ← 6. Apply Weights
         ↓
7. Generate HTML Report with Radar Chart
         ↓
8. Open in Browser (auto-opens)

Output Files

tests/reports/
├── testability-results-<timestamp>.json  # Raw data
├── testability-report-<timestamp>.html   # Visual report
└── latest.json                           # Symlink

Integration Examples

CI/CD Integration

# GitHub Actions
- name: Testability Assessment
  run: |
    timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh ${{ env.APP_URL }}

- name: Upload Reports
  uses: actions/upload-artifact@v3
  with:
    name: testability-reports
    path: tests/reports/testability-*.html

QX Partner Integration

// Combine testability with QX analysis
const qxAnalysis = await Task("QX Analysis", {
  target: 'https://example.com',
  integrateTestability: true
}, "qx-partner");

// Returns combined insights:
// - QX Score: 78/100
// - Testability Integration: Observability 72/100
// - Combined Insight: Low observability may mask UX issues

Programmatic Usage

import { runTestabilityAssessment } from './testability';

const results = await runTestabilityAssessment('https://example.com');
console.log(`Overall: ${results.overallScore}/100 (${results.grade})`);
console.log('Recommendations:', results.recommendations);

Agent Integration

// Run testability assessment
const assessment = await Task("Testability Assessment", {
  url: 'https://example.com',
  generateReport: true,
  openBrowser: true
}, "qe-quality-analyzer");

// Use with QX Partner for holistic analysis
const qxReport = await Task("Full QX Analysis", {
  target: 'https://example.com',
  integrateTestability: true,
  detectOracleProblems: true
}, "qx-partner");

Vibium Integration (Optional)

Overview

Vibium browser automation can be used alongside Playwright for enhanced testability assessment. While Playwright remains the primary engine, Vibium offers complementary capabilities for certain metrics.

Installation:

claude mcp add vibium -- npx -y vibium

Vibium-Enhanced Metrics

PrincipleVibium EnhancementBenefit
ObservabilityAuto-wait duration trackingMeasures DOM stability (30s timeout, 100ms polling)
ControllabilityElement interaction success rateValidates automation readiness via MCP
StabilityScreenshot consistencyVisual regression detection for layout stability
ExplainabilityElement attribute extractionARIA labels, semantic HTML validation

When to Use Vibium

✅ USE Vibium for:

  • Element stability metrics (auto-wait duration analysis)
  • Visual consistency checks (screenshot comparison)
  • MCP-native AI agent integration
  • Lightweight Docker images (400MB vs 1.2GB)

❌ USE Playwright for:

  • Console error detection (Vibium V1 lacks console API)
  • Network performance metrics (BiDi network APIs coming in V2)
  • Comprehensive browser coverage (Firefox, Safari)
  • Production-proven stability (Vibium V1 released Dec 2024)

Hybrid Assessment Example

// Testability assessment using both engines
const assessment = {
  // Playwright: Comprehensive metrics
  playwright: await runPlaywrightAssessment(url),

  // Vibium: Stability metrics
  vibium: {
    elementStability: await measureAutoWaitDuration(url),
    visualConsistency: await compareScreenshots(url),
    accessibilityAttributes: await extractARIALabels(url)
  }
};

// Enhanced Observability Score
const observability =
  (assessment.playwright.consoleErrors * 0.6) +
  (assessment.vibium.elementStability * 0.4);

Vibium MCP Tools for Testability

// 1. Element Stability Measurement
const browser = await browser_launch();
await browser_navigate({ url });
const startTime = Date.now();
const element = await browser_find({ selector: ".critical-element" });
const autoWaitDuration = Date.now() - startTime;
// Lower duration = better stability

// 2. Visual Consistency Check
const screenshot1 = await browser_screenshot();
await browser_navigate({ url }); // Reload
const screenshot2 = await browser_screenshot();
const visualDiff = compareImages(screenshot1.png, screenshot2.png);
// Lower diff = better stability

// 3. Accessibility Attribute Extraction
const elements = await browser_find({ selector: "button, a, input" });
const ariaLabels = elements.map(el => el.attributes["aria-label"]);
const semanticScore = (ariaLabels.filter(Boolean).length / elements.length) * 100;

Migration Strategy

Current (V2.2): Hybrid approach

  • Playwright: Primary engine for all 10 principles
  • Vibium: Optional enhancement for stability metrics

Future (V3.0): When Vibium V2 ships

  • Evaluate Vibium as primary engine if:
    • Console/Network APIs available
    • Production stability proven
    • Community adoption increases

Agent Coordination Hints

Memory Namespace

aqe/testability/
├── assessments/*       - Assessment results by URL
├── historical/*        - Historical scores for trend analysis
├── recommendations/*   - Improvement recommendations
├── integration/*       - QX integration data
└── vibium/*           - Vibium-specific metrics (optional)

Fleet Coordination

const testabilityFleet = await FleetManager.coordinate({
  strategy: 'testability-assessment',
  agents: [
    'qe-quality-analyzer',  // Primary assessment
    'qx-partner',           // UX integration
    'qe-visual-tester'      // Visual validation
  ],
  topology: 'sequential'
});

Common Issues & Solutions

IssueSolution
Tests timing outIncrease timeout: timeout 300 ./scripts/run-assessment.sh URL
Partial resultsCheck console errors, increase network timeout
Report not openingUse AUTO_OPEN=false, open manually
Config not updatingUse TEST_URL env var instead
Vibium not availableInstall via claude mcp add vibium -- npx -y vibium (optional)
Hybrid mode errorsVibium is optional; assessments work without it

Related Skills

  • accessibility-testing - WCAG compliance (overlaps with Explainability)
  • visual-testing-advanced - UI consistency
  • performance-testing - Load time metrics

Credits & References

Framework Origin

Implementation

Vibium Resources


Remember

Testability is an investment, not an afterthought.

Good testability:

  • Reduces debugging time
  • Enables faster feedback loops
  • Makes defects easier to find
  • Supports continuous testing

Low scores = High risk. Prioritize improvements by weight × impact.

FAQ

What does the testability-scoring skill do?

AI-powered testability assessment using 10 principles of intrinsic testability with Playwright and optional Vibium integration. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports.

How do I install testability-scoring?

Run npx skills add https://github.com/proffesor-for-testing/agentic-qe --skill testability-scoring in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/testability-scoring/ (personal) or .claude/skills/testability-scoring/ (project).

Which AI agents can use testability-scoring?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is testability-scoring safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains testability-scoring?

It is published by proffesor-for-testing in proffesor-for-testing/agentic-qe. It has 114 recorded installs.

More from proffesor-for-testing/agentic-qe

View all →

code-review-quality

proffesor-for-testing/agentic-qe

Conduct context-driven code reviews focusing on quality, testability, and maintainability. Use when reviewing code, providing feedback, or establishing review practices.

compatibility-testing

proffesor-for-testing/agentic-qe

Cross-browser, cross-platform, and cross-device compatibility testing ensuring consistent experience across environments. Use when validating browser support, testing responsive design, or ensuring platform compatibility.

Testing & QA652 installs

api-testing-patterns

proffesor-for-testing/agentic-qe

Comprehensive API testing patterns including contract testing, REST/GraphQL testing, and integration testing. Use when testing APIs or designing API test strategies.

Testing & QA606 installs

regression-testing

proffesor-for-testing/agentic-qe

Strategic regression testing with test selection, impact analysis, and continuous regression management. Use when verifying fixes don't break existing functionality, planning regression suites, or optimizing test execution for faster feedback.

Testing & QA331 installs

test-automation-strategy

proffesor-for-testing/agentic-qe

Design and implement effective test automation with proper pyramid, patterns, and CI/CD integration. Use when building automation frameworks or improving test efficiency.

Testing & QA254 installs

github-code-review

proffesor-for-testing/agentic-qe

Comprehensive GitHub code review with AI-powered swarm coordination

Popular testing & qa skills

View category →

tdd

mattpocock/skills

Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.

Testing & QA1.2M installs

grilling

mattpocock/skills

Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.

Testing & QA996.2K installs

migrate-to-shoehorn

mattpocock/skills

Migrate test files from as type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace as in tests, or needs partial test data.

Testing & QA484.1K installs

run-train

lllllllama/rigorpilot-skills

Rigor Train skill for deep learning research repositories. Use when a documented or selected training command should be run conservatively for startup verification, short-run verification, full kickoff, or resume, with command, config, seed, log, checkpoint, status, and metric evidence written to standardized trainoutputs/. Do not use for environment setup, exploratory sweeps, speculative idea implementation, or end-to-end orchestration.

Testing & QA337.8K installs

test-driven-development

obra/superpowers

Use when implementing any feature or bugfix, before writing implementation code

Testing & QA261.8K installs

Use when implementation is complete, all tests pass, and you need to decide how to integrate the work

Testing & QA216.8K installs