reproduce
End-to-end paper reproduction from arxiv URL through smoke runs to replication experiments. Handles missing or partial official code, missing training scripts, missing hyperparameters, and private datasets via similar-public-dataset substitution. Use when the user asks to reproduce, implement, replicate, or re-run a paper from scratch, or pastes an arxiv URL with reproduction intent.
- Installs
- 84
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/fcakyon/phd-skills --skill reproduce- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads reproduce automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Reproduce: paper reproduction from scratch
Reproducing an ML paper often means filling gaps the authors didn't ship, training scripts, hyperparameter tables, augmentation specifics, exact dataset splits. This skill walks seven stages from "I have an arxiv link" to "I have a replication run with measurable delta vs the paper's number."
Each stage has a separate reference file under references/ so this overview stays scannable.
When to run
The user just said any of:
- "reproduce / implement / replicate / re-run paper X"
- pasted an arxiv URL with reproduction intent ("can you redo this", "let's try this approach")
- pointed at an OpenReview / proceedings link with the same intent
- said "the paper has no code, can we build it"
Workflow
| Stage | What | Reference |
|---|---|---|
| 1 | Paper acquisition (CLI + PDF → structured extract) | references/01-paper-fetch.md |
| 2 | Existing code discovery + inventory | references/02-code-clone.md |
| 3 | Gap analysis (extract every missing hyperparam from the prose) | references/03-gap-analysis.md |
| 4 | Implementation (uv venv, fill gaps, commit per gap) | references/04-implement.md |
| 5 | Dataset acquisition (HF datasets first; substitute if private) | references/05-dataset.md |
| 6 | Smoke runs (forward pass → 1 step → 20 iters) | references/06-smoke.md |
| 7 | Replication runs + comparison at paper's reported epochs | references/07-replicate.md |
Walk them in order. Each stage has its own success criteria; do not advance to the next until the current one passes.
Working directory layout
For each paper reproduction, set up a dedicated workspace:
repro/<paper-arxiv-id>/
├── paper.md # structured extract from stage 1
├── inventory.md # what exists / missing from stage 2
├── gaps_filled.md # hyperparam table with provenance from stage 3
├── code/ # implementation from stage 4 (or cloned + extended)
├── data/ # dataset symlinks or actual data from stage 5
├── dataset_substitution.md # if a public dataset stood in for a private one
├── smoke_logs/ # outputs from stage 6
└── results.md # replication outcomes from stage 7
This keeps reproductions self-contained and easy to revisit later.
Cross-references
- After stage 3, hand the gap analysis off to the
paper-verificationskill for a round-trip check ("did I really capture every hyperparam the paper mentions"). - Stage 4 implementation should be committed in small, reviewable pieces: each commit references the paper section that justified the filled value.
- Stage 6 smoke failures route to the
/phd-skills:debugskill, not to ad-hoc fixes. - Stage 7 launches go through the
/phd-skills:launchchecklist before any multi-hour run. - Stage 7 comparisons go through the
/phd-skills:compareskill at the paper's reported epochs (never current-vs-final).
Output
For each reproduction, the final artifact is results.md with absolute deltas (not just %) and one of three labels per metric:
[matched within 0.X pp]: within the paper's reported variance[gap, hypothesis: ...]: measurable underperformance, with a stated hypothesis for the cause[fundamental disagreement, see X]: the result and the paper's claim are inconsistent in a way that needs investigation, not just more compute
If the workspace is on a public repo, link the workspace README from the project's main reproduction-tracking doc.
FAQ
What does the reproduce skill do?
End-to-end paper reproduction from arxiv URL through smoke runs to replication experiments. Handles missing or partial official code, missing training scripts, missing hyperparameters, and private datasets via similar-public-dataset substitution. Use when the user asks to reproduce, implement, replicate, or re-run a paper from scratch, or pastes an arxiv URL with reproduction intent.
How do I install reproduce?
Run npx skills add https://github.com/fcakyon/phd-skills --skill reproduce in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/reproduce/ (personal) or .claude/skills/reproduce/ (project).
Which AI agents can use reproduce?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is reproduce safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains reproduce?
It is published by fcakyon in fcakyon/phd-skills. It has 84 recorded installs.
More from fcakyon/phd-skills
paper-writing
fcakyon/phd-skills
Use when the user wants to write, structure, or revise academic paper sections, improve notation consistency, or refine figures and tables. Triggers on phrases like "write the abstract", "structure the methods", "improve this section", "notation consistency", "figure refinement", or "paper structure".
literature-research
fcakyon/phd-skills
Use when the user wants to find related work, survey a research area, identify literature gaps, or discover open-source implementations. Triggers on phrases like "find papers on", "related work", "literature review", "what papers exist", "open source implementation", or "papers with code".
paper-verification
fcakyon/phd-skills
Use when the user wants to verify paper claims against code or data, audit numerical accuracy, check formula-code alignment, or validate citation accuracy. Triggers on phrases like "verify claims", "check numbers", "do the numbers match", "formula vs code", "audit the paper", or "cross-check results".
experiment-design
fcakyon/phd-skills
Use when the user wants to design experiments, plan ablation studies, structure baselines, or create incremental evaluation strategies. Triggers on phrases like "design ablation", "plan experiment", "what experiments should I run", "baseline comparison", or "experiment matrix".
reviewer-defense
fcakyon/phd-skills
Use when the user wants to anticipate reviewer questions, select the strongest ablations to present, prepare rebuttals, or identify paper weaknesses before submission. Triggers on phrases like "reviewer questions", "anticipate reviewers", "rebuttal", "paper weaknesses", "defend the paper", or "strengthen the paper".
latex-setup
fcakyon/phd-skills
Use when the user wants to set up or troubleshoot a LaTeX environment, choose between biber and bibtex, install packages for a specific venue template, or configure compilation. Triggers on phrases like "setup latex", "biber vs bibtex", "latex compilation error", "install latex packages", "venue template", or "texlive setup".
Popular education & science skills
teach
mattpocock/skills
Teach the user a new skill or concept, within this workspace.
scaffold-exercises
mattpocock/skills
Create exercise directory structures with sections, problems, solutions, and explainers that pass linting. Use when user wants to scaffold exercises, create exercise stubs, or set up a new course section.
paper-context-resolver
lllllllama/rigorpilot-skills
Rigor Paper Context helper for README-first deep learning repo reproduction. Use only when the README and repository files leave a narrow reproduction-critical gap and the task is to resolve a specific paper detail such as dataset split, preprocessing, evaluation protocol, checkpoint mapping, or runtime assumption from primary paper sources while recording conflicts. Do not use for general paper summary, repo scanning, environment setup, command execution, title-only paper lookup, or replacing README guidance by default.
caveman-learn
juliusbrussee/caveman
Act on a Caveman learn report - review the ranked token sinks, apply cost-lowering fixes with per-edit consent, and report what those fixes returned. Use when asked to lower an agent's token cost, what caveman has saved, to trim a heavy CLAUDE.md, or to offload re-pasted context into cavemem.
gsap-plugins
greensock/gsap-skills
Official GSAP skill for GSAP plugins — registration, ScrollToPlugin, ScrollSmoother, Flip, Draggable, Inertia, Observer, SplitText, ScrambleText, SVG and physics plugins, CustomEase, EasePack, CustomWiggle, CustomBounce, GSDevTools. Use when the user asks about a GSAP plugin, scroll-to, flip animations, draggable, SVG drawing, or plugin registration.
teach-impeccable
pbakaus/impeccable
DEPRECATED: Use /impeccable teach instead. This command has been folded into the impeccable skill.