bayesian-reasoning-calibration
Applies Bayesian reasoning to systematically update probability estimates with new evidence, helping make better forecasts and avoid overconfidence. Use when making predictions or judgments under uncertainty, forecasting outcomes, evaluating probabilities, testing hypotheses, calibrating confidence, assessing risks with uncertain data, or when user mentions priors, likelihoods, Bayes theorem, probability updates, forecasting, calibration, or belief revision.
- Installs
- 156
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/lyndonkl/claude --skill bayesian-reasoning-calibration- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads bayesian-reasoning-calibration automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Bayesian Reasoning & Calibration
Table of Contents
Core formula: P(H|E) = P(E|H) x P(H) / P(E), where P(H) = prior, P(E|H) = likelihood, P(H|E) = posterior.
Quick Example:
# Should we launch Feature X?
## Prior Belief
Before beta testing: 60% chance of adoption >20%
- Base rate: Similar features get 15-25% adoption
- Our feature seems stronger than average
- Prior: 60%
## New Evidence
Beta test: 35% of users adopted (70 of 200 users)
## Likelihoods
If true adoption is >20%:
- P(seeing 35% in beta | adoption >20%) = 75% (likely to see high beta if true)
If true adoption is ≤20%:
- P(seeing 35% in beta | adoption ≤20%) = 15% (unlikely to see high beta if false)
## Bayesian Update
Posterior = (75% × 60%) / [(75% × 60%) + (15% × 40%)]
Posterior = 45% / (45% + 6%) = 88%
## Conclusion
Updated belief: 88% confident adoption will exceed 20%
Evidence strongly supports launch, but not certain.
Workflow
Copy this checklist and track your progress:
Bayesian Reasoning Progress:
- [ ] Step 1: Define the question
- [ ] Step 2: Establish prior beliefs
- [ ] Step 3: Identify evidence and likelihoods
- [ ] Step 4: Calculate posterior
- [ ] Step 5: Calibrate and document
Step 1: Define the question
Clarify hypothesis (specific, testable claim), probability to estimate, timeframe (when outcome is known), success criteria, and why this matters (what decision depends on it). Example: "Product feature will achieve >20% adoption within 3 months" - matters for launch decision.
Step 2: Establish prior beliefs
Set initial probability using base rates (general frequency), reference class (similar situations), specific differences, and explicit probability assignment with justification. Good priors are based on base rates, account for differences, honest about uncertainty, and include ranges if unsure (e.g., 40-60%). Avoid purely intuitive priors, ignoring base rates, or extreme values without justification.
Step 3: Identify evidence and likelihoods
Assess evidence (specific observation/data), diagnostic power (does it distinguish hypotheses?), P(E|H) (probability if hypothesis TRUE), P(E|¬H) (probability if FALSE), and calculate likelihood ratio = P(E|H) / P(E|¬H). LR > 10 = very strong evidence, 3-10 = moderate, 1-3 = weak, ≈1 = not diagnostic, <1 = evidence against.
Step 4: Calculate posterior
Apply Bayes' Theorem: P(H|E) = [P(E|H) × P(H)] / P(E), or use odds form: Posterior Odds = Prior Odds × Likelihood Ratio. Calculate P(E) = P(E|H)×P(H) + P(E|¬H)×P(¬H), get posterior probability, and interpret change. For simple cases → Use resources/template.md calculator. For complex cases (multiple hypotheses) → Study resources/methodology.md.
Step 5: Calibrate and document
Check calibration (over/underconfident?), validate assumptions (are likelihoods reasonable?), perform sensitivity analysis, create bayesian-reasoning-calibration.md, and note limitations. Self-check using resources/evaluators/rubric_bayesian_reasoning_calibration.json: verify prior based on base rates, likelihoods justified, evidence diagnostic (LR ≠ 1), calculation correct, posterior calibrated, assumptions stated, sensitivity noted. Minimum standard: Score ≥ 3.5.
Common Patterns
For forecasting:
- Use base rates as starting point
- Update incrementally as evidence arrives
- Track forecast accuracy over time
- Calibrate by comparing predictions to outcomes
For hypothesis testing:
- State competing hypotheses explicitly
- Calculate likelihood ratio for evidence
- Update belief proportionally to evidence strength
- Don't claim certainty unless LR is extreme
For risk assessment:
- Consider multiple scenarios (not just binary)
- Update risks as new data arrives
- Use ranges when uncertain about likelihoods
- Perform sensitivity analysis
For avoiding bias:
- Force explicit priors (prevents anchoring to evidence)
- Use reference classes (prevents ignoring base rates)
- Calculate mathematically (prevents motivated reasoning)
- Document before seeing outcome (enables calibration)
Guardrails
Do:
- State priors explicitly before seeing all evidence
- Use base rates and reference classes
- Estimate likelihoods with justification
- Update incrementally as evidence arrives
- Be honest about uncertainty
- Perform sensitivity analysis
- Track forecasts for calibration
- Acknowledge limits of the model
Don't:
- Use extreme priors (1%, 99%) without exceptional justification
- Ignore base rates (common bias)
- Treat all evidence as equally diagnostic
- Update to 100% certainty (almost never justified)
- Cherry-pick evidence
- Skip documenting reasoning
- Forget to calibrate (compare predictions to outcomes)
- Apply to questions where probability is meaningless
Quick Reference
- Standard template:
resources/template.md - Multiple hypotheses:
resources/methodology.md - Examples:
resources/examples/product-launch.md,resources/examples/medical-diagnosis.md - Quality rubric:
resources/evaluators/rubric_bayesian_reasoning_calibration.json
Bayesian Formula (Odds Form):
Posterior Odds = Prior Odds × Likelihood Ratio
Likelihood Ratio:
LR = P(Evidence | Hypothesis True) / P(Evidence | Hypothesis False)
Output naming: bayesian-reasoning-calibration.md or {topic}-forecast.md
FAQ
What does the bayesian-reasoning-calibration skill do?
Applies Bayesian reasoning to systematically update probability estimates with new evidence, helping make better forecasts and avoid overconfidence. Use when making predictions or judgments under uncertainty, forecasting outcomes, evaluating probabilities, testing hypotheses, calibrating confidence, assessing risks with uncertain data, or when user mentions priors, likelihoods, Bayes theorem, probability updates, forecasting, calibration, or belief revision.
How do I install bayesian-reasoning-calibration?
Run npx skills add https://github.com/lyndonkl/claude --skill bayesian-reasoning-calibration in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/bayesian-reasoning-calibration/ (personal) or .claude/skills/bayesian-reasoning-calibration/ (project).
Which AI agents can use bayesian-reasoning-calibration?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is bayesian-reasoning-calibration safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains bayesian-reasoning-calibration?
It is published by lyndonkl in lyndonkl/claude. It has 156 recorded installs.
More from lyndonkl/claude
evaluation-rubrics
lyndonkl/claude
Designs structured scoring tools with explicit criteria, performance scales, and descriptors for consistent, transparent quality assessment. Use when need quality criteria and scoring scales to evaluate work consistently, compare alternatives objectively, set acceptance thresholds, reduce subjective bias, or when user mentions rubric, scoring criteria, quality standards, evaluation framework, inter-rater reliability, or grading/assessing work.
strategy-and-competitive-analysis
lyndonkl/claude
Develops business strategies grounded in rigorous competitive and market analysis using proven frameworks (Good Strategy kernel, Porter's 5 Forces, SWOT, Blue Ocean, Playing to Win, Value Chain Analysis, BCG Matrix). Use when developing strategy (market entry, product launch, expansion, M&A, turnaround), conducting competitive analysis, making strategic decisions (build vs buy, pricing, positioning), planning strategic initiatives, or when user mentions strategy, competitive analysis, Porter's 5 Forces, SWOT, market positioning, or strategic frameworks.
scientific-manuscript-review
lyndonkl/claude
Guides systematic multi-pass review and editing of scientific manuscripts (research articles, reviews, perspectives) to improve clarity, structure, scientific rigor, and reader comprehension. Use when reviewing or editing research manuscripts, journal articles, or perspectives, when user mentions manuscript, paper draft, article, research writing, journal submission, reviewer feedback, or needs to improve scientific writing.
memory-retrieval-learning
lyndonkl/claude
Creates evidence-based learning plans that maximize long-term retention through spaced repetition, retrieval practice, interleaving, and elaboration. Guides through goal definition, material breakdown, review scheduling, and progress tracking. Use when long-term knowledge retention is needed, studying for exams or certifications, learning new job skills or technology, mastering substantial material, combating forgetting, or when user mentions studying, memorizing, learning plans, spaced repetition, flashcards, active recall, or durable learning.
socratic-teaching-scaffolds
lyndonkl/claude
Guides learners to discover knowledge through strategic Socratic questioning and progressive scaffolding removal. Combines question ladders, misconception detectors, Feynman explanations, and worked-example fading to build durable understanding. Use when teaching complex concepts, correcting misconceptions, onboarding team members, mentoring problem-solving, or designing self-paced learning. Use when user mentions "teach me", "help me understand", "explain like I'm", "learning path", "guided discovery", or "Socratic method".
grant-proposal-assistant
lyndonkl/claude
Guides creation and review of competitive grant proposals (NIH R01/R21/K, NSF, foundations) by applying reviewer-perspective thinking to ensure clear hypotheses, compelling significance, genuine innovation, and feasible approaches. Use when writing or reviewing grant proposals, crafting specific aims, drafting significance/innovation/approach sections, or when user mentions R01, R21, K-series, grant writing, proposal review, study section, or fundable hypothesis.
Popular testing & qa skills
tdd
mattpocock/skills
Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
grilling
mattpocock/skills
Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
migrate-to-shoehorn
mattpocock/skills
Migrate test files from as type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace as in tests, or needs partial test data.
run-train
lllllllama/rigorpilot-skills
Rigor Train skill for deep learning research repositories. Use when a documented or selected training command should be run conservatively for startup verification, short-run verification, full kickoff, or resume, with command, config, seed, log, checkpoint, status, and metric evidence written to standardized trainoutputs/. Do not use for environment setup, exploratory sweeps, speculative idea implementation, or end-to-end orchestration.
test-driven-development
obra/superpowers
Use when implementing any feature or bugfix, before writing implementation code
finishing-a-development-branch
obra/superpowers
Use when implementation is complete, all tests pass, and you need to decide how to integrate the work