MCPInstall

model-evaluation-benchmark

by rysweetTesting & QA

Automated reproduction of comprehensive model evaluation benchmarks following the Benchmark Suite V3. Auto-activates for model benchmarking, comparison evaluation, or performance testing between AI models.

Installs
159
Last 24h
—
Trending rank
—
Security
Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/rysweet/amplihack --skill model-evaluation-benchmark
  1. 01

    Install

    Run the command in your project. The CLI asks which agents to install for.

  2. 02

    Describe the task

    Ask your agent for something this skill covers. It loads model-evaluation-benchmark automatically when relevant.

  3. 03

    Review the result

    The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

Model Evaluation Benchmark Skill

Purpose: Automated reproduction of comprehensive model evaluation benchmarks following the Benchmark Suite V3 reference implementation.

Auto-activates when: User requests model benchmarking, comparison evaluation, or performance testing between AI models in agentic workflows.

Skill Description

This skill orchestrates end-to-end model evaluation benchmarks that measure:

  • Efficiency: Duration, turns, cost, tool calls
  • Quality: Code quality scores via reviewer agents
  • Workflow Adherence: Subagent calls, skills used, workflow step compliance
  • Artifacts: GitHub issues, PRs, documentation generated

The skill automates the entire benchmark workflow from execution through cleanup, following the v3 reference implementation.

When to Use

✅ Use when:

  • Comparing AI models (Opus vs Sonnet, etc.)
  • Measuring workflow adherence
  • Generating comprehensive benchmark reports
  • Need reproducible benchmarking

❌ Don't use when:

  • Simple code reviews (use reviewer)
  • Performance profiling (use optimizer)
  • Architecture decisions (use architect)

Execution Instructions

When this skill is invoked, follow these steps:

Phase 1: Setup

  1. Read tests/benchmarks/benchmark_suite_v3/BENCHMARK_TASKS.md
  2. Identify models to benchmark (default: Opus 4.5, Sonnet 4.5)
  3. Create TodoWrite list with all phases

Phase 2: Execute Benchmarks

For each task × model:

cd tests/benchmarks/benchmark_suite_v3
python run_benchmarks.py --model {opus|sonnet} --tasks 1,2,3,4

Phase 3: Analyze Results

  1. Read all result files: ~/.amplihack/.claude/runtime/benchmarks/suite_v3/*/result.json
  2. Launch parallel Task tool calls with subagent_type="reviewer" to:
    • Analyze trace logs for tool/agent/skill usage
    • Score code quality (1-5 scale)
  3. Synthesize findings

Phase 4: Generate Report

  1. Create markdown report following BENCHMARK_REPORT_V3.md structure
  2. Create GitHub issue with report
  3. Archive artifacts to GitHub release
  4. Update issue with release link

Phase 5: Cleanup (MANDATORY)

  1. Close all benchmark PRs: gh pr close {numbers}
  2. Close all benchmark issues: gh issue close {numbers}
  3. Remove worktrees: git worktree remove worktrees/bench-*
  4. Verify cleanup complete

See tests/benchmarks/benchmark_suite_v3/CLEANUP_PROCESS.md for detailed cleanup instructions.

Example Usage

User: "Run model evaluation benchmark"Assistant: I'll run the complete benchmark suite following the v3 reference implementation.

[Executes phases 1-5 above]

Final Report: See GitHub Issue #XXXX
Artifacts: https://github.com/.../releases/tag/benchmark-suite-v3-artifacts

References

  • Reference Report: tests/benchmarks/benchmark_suite_v3/BENCHMARK_REPORT_V3.md
  • Task Definitions: tests/benchmarks/benchmark_suite_v3/BENCHMARK_TASKS.md
  • Cleanup Guide: tests/benchmarks/benchmark_suite_v3/CLEANUP_PROCESS.md
  • Runner Script: tests/benchmarks/benchmark_suite_v3/run_benchmarks.py

Last Updated: 2025-11-26 Reference Implementation: Benchmark Suite V3 GitHub Issue Example: #1698

FAQ

What does the model-evaluation-benchmark skill do?

Automated reproduction of comprehensive model evaluation benchmarks following the Benchmark Suite V3. Auto-activates for model benchmarking, comparison evaluation, or performance testing between AI models.

How do I install model-evaluation-benchmark?

Run npx skills add https://github.com/rysweet/amplihack --skill model-evaluation-benchmark in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/model-evaluation-benchmark/ (personal) or .claude/skills/model-evaluation-benchmark/ (project).

Which AI agents can use model-evaluation-benchmark?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is model-evaluation-benchmark safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains model-evaluation-benchmark?

It is published by rysweet in rysweet/amplihack. It has 159 recorded installs.

More from rysweet/amplihack

View all →

cybersecurity-analyst

rysweet/amplihack

Analyzes events through cybersecurity lens using threat modeling, attack surface analysis, defense-in-depth, zero-trust architecture, and risk-based frameworks (CIA triad, STRIDE, MITRE ATT&CK). Provides insights on vulnerabilities, attack vectors, defense strategies, incident response, and security posture. Use when: Security incidents, vulnerability assessments, threat analysis, security architecture, compliance. Evaluates: Confidentiality, integrity, availability, threat actors, attack patterns, controls, residual risk.

Security1.5K installs

lawyer-analyst

rysweet/amplihack

Analyzes events through legal lens using statutory interpretation, case law analysis, legal reasoning, constitutional principles, and multiple legal frameworks (common law, civil law, international law). Provides insights on legal rights, obligations, liabilities, remedies, and compliance requirements. Use when: Legal disputes, contracts, regulations, compliance, rights analysis, liability assessment. Evaluates: Legal obligations, rights, liabilities, remedies, precedent, statutory authority, constitutionality.

Business & Finance688 installs

psychologist-analyst

rysweet/amplihack

Analyzes events through psychological lens using cognitive psychology, social psychology, developmental psychology, clinical psychology, and neuroscience. Provides insights on behavior, cognition, emotion, motivation, group dynamics, decision-making biases, mental health, and individual differences. Use when: Behavioral patterns, decision-making, group behavior, mental health, leadership, persuasion, trauma, development. Evaluates: Cognitive processes, emotional responses, motivations, biases, group dynamics, personality, mental states.

Education & Science450 installs

mermaid-diagram-generator

rysweet/amplihack

Converts architecture descriptions, module specs, or workflow docs into Mermaid diagrams. Use when visualizing brick module relationships, workflows (DDD, investigation), or system architecture. Supports: flowcharts, sequence diagrams, class diagrams, state machines, entity relationship diagrams, and Gantt charts. Generates valid Mermaid syntax for embedding in markdown docs.

Docs & Writing428 installs

pptx

rysweet/amplihack

Presentation creation, editing, and analysis. When Claude needs to work with presentations (.pptx files) for: (1) Creating new presentations, (2) Modifying or editing content, (3) Working with layouts, (4) Adding comments or speaker notes, or any other presentation tasks

Docs & Writing420 installs

economist-analyst

rysweet/amplihack

Analyzes events through economic lens using supply/demand, incentive structures, market dynamics, and multiple schools of economic thought (Classical, Keynesian, Austrian, Behavioral). Provides insights on market impacts, resource allocation, policy implications, and distributional effects. Use when: Economic events, policy changes, market shifts, financial crises, regulatory decisions. Evaluates: Incentives, efficiency, opportunity costs, market failures, systemic risks.

Business & Finance378 installs

Popular testing & qa skills

View category →

tdd

mattpocock/skills

Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.

Testing & QA1.2M installs

grilling

mattpocock/skills

Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.

Testing & QA996.2K installs

migrate-to-shoehorn

mattpocock/skills

Migrate test files from as type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace as in tests, or needs partial test data.

Testing & QA484.1K installs

run-train

lllllllama/rigorpilot-skills

Rigor Train skill for deep learning research repositories. Use when a documented or selected training command should be run conservatively for startup verification, short-run verification, full kickoff, or resume, with command, config, seed, log, checkpoint, status, and metric evidence written to standardized trainoutputs/. Do not use for environment setup, exploratory sweeps, speculative idea implementation, or end-to-end orchestration.

Testing & QA337.8K installs

test-driven-development

obra/superpowers

Use when implementing any feature or bugfix, before writing implementation code

Testing & QA261.8K installs

finishing-a-development-branch

obra/superpowers

Use when implementation is complete, all tests pass, and you need to decide how to integrate the work

Testing & QA216.8K installs