mandela
Audit any eval, metric, experiment, or benchmark for leakage — does external ground-truth enter independently, or are the model, scorer, and designer just confirming a result no outside truth ever produced? Use before trusting any 'how we'll know it worked' — an A/B, a holdout, a score, a validation — and whenever a result feels too clean or self-confirming. Walks an 8-pattern leakage taxonomy and returns only the patterns that fire, each with an independence fix. Read-only.
- Installs
- 1.3K
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/lilmgenius/paperthin --skill mandela- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads mandela automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Audit a validation for leakage: does outside ground-truth actually enter, or is everyone confirming a result no one independently produced?
Goal
The name is the Mandela Effect — a whole population confidently remembers something that never independently happened; a leaky validation is the same shape. Walk the 8 patterns below. mandela checks one thing: whether a validation is independent, or whether the designer, model, and scorer are only confirming each other.
Workflow
- Identify the validation (eval / metric / experiment / holdout / "how we'll know"). Name its components — what plays model, scorer, designer, dataset.
- Ask the core question: does external ground-truth enter independently?
- Test the validation against all 8 patterns below (some apply only to certain components — a human subject, a scorer); report only the ones that fire, each by name.
- Give the independent-ground-truth fix for each hit.
The 8 leakage patterns
- Recall, not reason — a memorized answer recited instead of one actually derived; the system already knows the result it is supposedly computing.
- Wrong null hypothesis — an ablation that removes a surface label but not the underlying signal the system actually exploits, so the "control" still leaks.
- Shared hallucination — two components verifying each other; circularity reported as a number.
- Tautology — a scorer grading buckets it drew itself.
- Verifier = designer — a private, unreproducible recipe in a holdout's clothes.
- Shared-pool bias — train and holdout drawn from one labeler pool, so one bias enters both sides.
- Frame injection — a question that hands the subject the hypothesis.
- Demand characteristics — measured subjects who know they're being measured.
Rules
- Subtlety that bites twice: you can blind the output value and still leak the collection recipe.
- Read-only — name the leak and the independence fix; don't rewrite the experiment.
- For a high-stakes validation, you may add one independent fresh-context auditor (
N=1) handed only the validation design, not this session's reasoning, to re-run the 8-pattern taxonomy blind; the default remains same-session and read-only.
Verification
Turn mandela on this audit:
- Run patterns #3–#5 on yourself: are you a scorer grading buckets you drew (Tautology, #4)? is the verifier the designer (#5)? is your verdict a shared hallucination with the design's own claims (#3)?
- Could a reader who didn't run the audit reach your verdict from the cited evidence alone — independent ground-truth?
- The report names the root, not a laundry list.
FAQ
What does the mandela skill do?
Audit any eval, metric, experiment, or benchmark for leakage — does external ground-truth enter independently, or are the model, scorer, and designer just confirming a result no outside truth ever produced? Use before trusting any 'how we'll know it worked' — an A/B, a holdout, a score, a validation — and whenever a result feels too clean or self-confirming. Walks an 8-pattern leakage taxonomy and returns only the patterns that fire, each with an independence fix. Read-only.
How do I install mandela?
Run npx skills add https://github.com/lilmgenius/paperthin --skill mandela in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/mandela/ (personal) or .claude/skills/mandela/ (project).
Which AI agents can use mandela?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is mandela safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains mandela?
It is published by lilmgenius in lilmgenius/paperthin. It has 1,317 recorded installs.
More from lilmgenius/paperthin
dedash
lilmgenius/paperthin
Remove em-dashes and the dashes standing in for them from a user-owned scope, reading each occurrence in context and choosing the punctuation or wording that fits.
ssotize
lilmgenius/paperthin
Audit and consolidate a fact that's scattered across places into one canonical source, after approval, and replace the rest with references. Use when asked to find duplication, check consistency, locate the source of truth, deduplicate, consolidate, unify, or establish SSOT across artifacts or platforms. Starts read-only, reports the map and plan, then mutates only after explicit approval.
shower
lilmgenius/paperthin
Cold-read the artifact you're focused on from a fresh, zero-context sub-session to confirm it stands on its own — a shower-thought reset for accumulated session bias. Use when a long session has worn away your fresh eyes and you can no longer tell whether the artifact in focus is clear to someone with no prior context; before a handoff, publish, or merge; or when you want a clean-room comprehension smoke test. Spawns a separate context-free reviewer; it diagnoses, it does not fix.
hate
lilmgenius/paperthin
Attack a plan, design, or argument like you want it to fail before you commit real effort — return the single load-bearing objection and the cheapest experiment that would prove it matters, not a checklist. User-invoked on purpose: a hate-it reflex always in the agent's reach would bias it toward demolition. Weighs many failure axes but owns the synthesis to one root.
factchk
lilmgenius/paperthin
Verify reality-grounded claims against external sources in both directions before they ship — could the 'absurd' be real, could the 'obvious' be false or long-established? Use whenever an artifact, or the sentence you are about to write, asserts something as plausible, realistic, absurd, novel, or impossible from intuition rather than a checked source; before relying on a factual claim in prose, a design rationale, a research claim, or a plan. Fires on metacognitive doubt — when you can't actually know, verify instead of trusting the feeling. Scans read-only, then fixes the clear errors or flags the judgment calls; leaves deliberate fiction alone.
re0
lilmgenius/paperthin
Refresh an existing artifact into the current best v0. Use when the user asks to clean up, sync up, dedupe, de-noise, smooth, rewrite, or update an artifact after iteration; when nearby artifacts may have drifted; or when changes in one place should be reflected across related artifacts while keeping the result minimal.
Popular frontend & design skills
setup-matt-pocock-skills
mattpocock/skills
Configure this repo for the engineering skills: set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.
frontend-design
anthropics/skills
Guidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.
hyperframes-animation
heygen-com/hyperframes
All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). Also covers auditing an existing composition's choreography (animation map) and 24 named text-animation effects. HyperFrames-native: single paused timeline, seek-safe, deterministic.
prototype
mattpocock/skills
Build a throwaway prototype to answer a design question. Use when the user wants to sanity-check whether a state model or logic feels right, or explore what a UI should look like.
hyperframes-core
heygen-com/hyperframes
The HyperFrames composition contract — build one renderable project. Use for composition structure, the data- timing attributes, class="clip", tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.
vercel-react-best-practices
vercel-labs/agent-skills
React and Next.js performance optimization guidelines from Vercel Engineering. This skill should be used when writing, reviewing, or refactoring React/Next.js code to ensure optimal performance patterns. Triggers on tasks involving React components, Next.js pages, data fetching, bundle optimization, or performance improvements.