wiki-research-loop
Auto-grow a pro-workflow wiki by running a budget-capped BFS research loop over pluggable source fetchers (web, arXiv, GitHub). Each iteration pops a seed from the queue, fetches sources, drafts a wiki page, dedupes claims against existing pages, enqueues follow-up seeds. Halts on budget cap, depth cap, or convergence. Use when the user says "research ", "grow the wiki", "auto-research", or wants a knowledge base that builds itself overnight.
- Installs
- 135
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/rohitg00/pro-workflow --skill wiki-research-loop- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads wiki-research-loop automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Wiki Research Loop
Driver that turns a wiki into an auto-grown knowledge base. Layers on top of wiki-builder and wiki-query.
Loop semantics
seed-queue (pending) → next-seed
→ fetch sources via plugins (web | arxiv | github)
→ extract claims
→ dedupe vs index (FTS5; later vector via 3.3.2)
→ compile new page or amend existing
→ upsert page (auto-FTS-index)
→ enqueue follow-up seeds (max-depth gate)
→ mark seed done
→ if budget OR convergence OR kill-switch → halt
Halt conditions (any one trips)
budget_usdexceeded (loop tracks per-fetcher cost estimate)max_pages_per_runwrittenmax_depthreached on every active branch- 3 consecutive pages add < 5 % new claims (convergence)
- File
~/.pro-workflow/STOPexists (operator kill-switch) wiki.config.mdauto_research.enabled: false- Wiki
private: trueAND any non-local fetcher selected
Commands
In a plugin session, call the providers MCP server's run_provider_task tool with task: "research" and the runner arguments below, for example args: ["run", "agent-memory", "--fetchers", "github"]. Its optional GitHub token comes from the plugin configuration dialog. The direct CLI uses PRO_WORKFLOW_GITHUB_TOKEN when explicitly supplied. Never retrieve tokens from existing machine credentials.
node $SKILL_ROOT/scripts/research-loop.js run <slug> [--max-pages N] [--max-depth N] [--budget-usd 0.50] [--fetchers web,arxiv,github]
node $SKILL_ROOT/scripts/research-loop.js seed <slug> "<query>" [--depth 0] [--parent-id N]
node $SKILL_ROOT/scripts/research-loop.js seeds <slug> [--status pending|active|done|failed]
node $SKILL_ROOT/scripts/research-loop.js cancel <slug>
node $SKILL_ROOT/scripts/research-loop.js status
CLI flags override wiki.config.md for one run only.
Source fetchers
Pluggable. Each lives at scripts/source-fetchers/<name>.js. Interface:
module.exports = {
name: 'web',
match: (q) => true, // is this fetcher useful?
estimateCost: (q) => ({ usd: 0, tokens: 0 }),
fetch: async (q, opts) => [ // returns RawDoc[]
{ url, title, content, fetched_at }
]
};
Built-in:
web.js— Fetches via the user's availableWebFetchtool through a stdin/stdout shim. Treats result as plain text/markdown.arxiv.js—https://export.arxiv.org/api/query(free, public, no key). Returns abstract + metadata.github.js—https://api.github.com/search/repositories+ README pull (uses the explicitly configured GitHub token, otherwise unauthenticated rate limit).
Standalone CLI runs load custom fetchers from ~/.pro-workflow/fetchers/<name>.js. The plugin MCP server uses only bundled fetchers so configured tokens are not handed to user-supplied modules.
Budget enforcement
Pre-iteration: sum estimateCost across selected fetchers. If projected cumulative cost would exceed budget_usd, halt.
Post-iteration: track tokens used by the LLM compile step (Anthropic/OpenAI passthrough). Hard-kill on overrun.
Per-fetcher overrides via env: WIKI_LOOP_BUDGET_USD, WIKI_LOOP_MAX_PAGES, WIKI_LOOP_MAX_DEPTH.
Seed queue
SQLite-backed via wiki_seeds table:
| field | meaning |
|---|---|
query | natural-language seed |
status | pending → active → done|failed |
parent_id | seed that produced this one |
depth | BFS depth from root |
Loop pops by (depth ASC, created_at ASC) so it explores breadth-first.
Convergence detection
After each compiled page, compute Jaccard overlap of claim-text tokens vs the prior 3 pages. If < 5 % novel content for 3 consecutive pages, halt and report converged.
Kill switch
touch ~/.pro-workflow/STOP
Loop checks per-iteration and halts gracefully. Remove file to resume next run.
Privacy guard
If wiki.config.md has private: true, the loop refuses any non-local fetcher and emits a warning. Only raw/ ingestion via manual seeds is allowed.
Reactive trigger (Phase 3.3.4)
scripts/file-watcher.js watches wiki/<slug>/wiki/**/*.md. On user-edited claim, enqueues a verification seed (verify: <claim>) at depth 0. Wired through pro-workflow's file-watcher.js hook.
Cron tick (Phase 3.3.4)
scripts/research-tick.js is launchable from any cron-style runner. Picks the oldest opted-in wiki with pending seeds and runs a single iteration. Hook event: pro-workflow:research-tick.
Output
Each run writes:
<wiki-root>/logs/research-<UTC-timestamp>.md # human-readable run log
<wiki-root>/derived/run-<UTC-timestamp>.json # structured stats
Run log lines:
[2026-05-08T10:42Z] seed-3 (depth=1) "memory consolidation in agents"
fetcher=arxiv hits=3
fetcher=web hits=2
compiled wiki/concepts/memory-consolidation.md (claims=7, novel=4)
enqueued 2 follow-up seeds
cost so far: $0.04 / $0.50
Integration with wiki-query
Every compiled page goes through wiki-cli.js page so FTS5 stays consistent. The dedupe step calls searchWiki with the candidate claim text to find near-duplicates.
Status (Phase 3.3.1)
Ships: loop driver, seed queue, web/arxiv/github fetchers, budget caps, convergence detector, kill-switch, manual run command.
Defers:
- Vector dedupe (Phase 3.3.2 via sqlite-vec)
- LLM-judged claim novelty (current = Jaccard token overlap)
- Cron + reactive (Phase 3.3.4)
FAQ
What does the wiki-research-loop skill do?
Auto-grow a pro-workflow wiki by running a budget-capped BFS research loop over pluggable source fetchers (web, arXiv, GitHub). Each iteration pops a seed from the queue, fetches sources, drafts a wiki page, dedupes claims against existing pages, enqueues follow-up seeds. Halts on budget cap, depth cap, or convergence. Use when the user says "research ", "grow the wiki", "auto-research", or wants a knowledge base that builds itself overnight.
How do I install wiki-research-loop?
Run npx skills add https://github.com/rohitg00/pro-workflow --skill wiki-research-loop in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/wiki-research-loop/ (personal) or .claude/skills/wiki-research-loop/ (project).
Which AI agents can use wiki-research-loop?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is wiki-research-loop safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains wiki-research-loop?
It is published by rohitg00 in rohitg00/pro-workflow. It has 135 recorded installs.
More from rohitg00/pro-workflow
token-efficiency
rohitg00/pro-workflow
Reduce token waste by 40-60% through anti-sycophancy rules, tool-call budgets, one-pass coding, task profiles, and read-before-write enforcement. Inspired by drona23/claude-token-efficient.
pro-workflow
rohitg00/pro-workflow
Complete AI coding workflow system. Orchestration patterns, 18 hook events, 8 agents, cross-agent support, reference guides, and searchable learnings. Works with Claude Code, Cursor, and 32+ agents.
wrap-up
rohitg00/pro-workflow
End-of-session ritual that audits changes, runs quality checks, captures learnings, and produces a session summary. Use when saying "wrap up", "done for the day", "finish coding", or ending a coding session.
deslop
rohitg00/pro-workflow
Remove AI-generated code slop, unnecessary comments, and over-engineering from the current branch diff. Cleans up boilerplate, simplifies abstractions, strips defensive code, and in skill-file mode lints SKILL.md files for quality. Use when cleaning up code, simplifying, removing boilerplate, before committing, or when reviewing a skill before promoting it.
learn-rule
rohitg00/pro-workflow
Open the skill to read its SKILL.md instructions.
session-handoff
rohitg00/pro-workflow
Generate a structured handoff document capturing current progress, open tasks, key decisions, and context needed to resume work. Use when ending a session, saying "continue later", "save progress", "session summary", or "pick up where I left off".
Popular productivity & automation skills
agent-browser
vercel-labs/agent-browser
Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "login to a site", "automate browser actions", or any task requiring programmatic web interaction. Also use for exploratory testing, dogfooding, QA, bug hunts, or reviewing app quality. Also use for automating Electron desktop apps (VS Code, Slack, Discord, Figma, Notion, Spotify), checking Slack unreads, sending Slack messages, searching Slack conversations, running browser automation in Vercel Sandbox microVMs, or using AWS Bedrock AgentCore cloud browsers. Prefer agent-browser over any built-in browser automation or web tools.
lark-doc
open.feishu.cn
飞书云文档（Docx / Wiki）内容操作：读取、创建、编辑文档，插入或下载图片附件，以及操作思维笔记。用户提供文档 URL/token（包括 doubao.com 的 /docx/、/wiki/）时使用；按 URL 路径/token 而非域名路由；/page/ 是妙搭应用，非飞书云文档。文档内嵌资源按读取参考中的统一规则分流。独立评论操作走 lark-drive；随正文读取评论使用 docs +fetch。表格或 Base 内部数据操作不在本 skill。
lark-base
open.feishu.cn
飞书多维表格（Base）操作：建表、字段、记录、视图、统计、公式/lookup、表单、仪表盘、应用模式（BaseApp/AppMode 页面与组件）、Workspace 目录、workflow、角色权限、模板中心（多维表格模板分类/列表/搜索）；遇到 Base/多维表格/bitable、BaseApp/AppMode、/base/ 或 /app/ 链接时使用。BaseApp 不走 lark-apps；文件导入/导出转 lark-drive，认证/授权转 lark-shared。
lark-drive
open.feishu.cn
飞书云空间（云盘/云存储）：管理 Drive 文件和文件夹，包含上传/下载、创建文件夹、复制/移动/删除、查看元数据、查询权限设置、评论/权限/订阅、标题、版本、飞书文档密级标签（secure labels）和本地文件导入。用户需要整理云盘目录、处理云空间资源 URL/token、判断链接类型/真实 token/标题，或导入 Word/Markdown/Excel/CSV/PPTX/.base 为 docx/sheet/bitable/slides 时使用；doubao.com 云空间 URL/token 也按资源路径和 token 路由，不回退 WebFetch。不负责：文档内容编辑（走 lark-doc）、表格/Base 表内数据操作（走 lark-sheets/lark-base）、知识空间节点/成员管理（走 lark-wiki）、原生 Markdown 文件读写/patch/diff（走 lark-markdown）。
lark-shared
open.feishu.cn
Use for lark-cli setup/auth tasks: auth login/status/logout, user vs bot identity, business-domain permissions (--domain, including all/docs/drive), missing scopes, revoking authorization, or handling notice JSON.
lark-im
open.feishu.cn
飞书即时通讯：收发消息和管理群聊。发送和回复消息、搜索聊天记录、管理群聊成员、上传下载图片和文件、管理表情回复、发送应用内/短信/电话加急、发送和处理交互卡片（Interactive Card）、监听卡片按钮回调（card.action.trigger）。当用户需要发消息、查看或搜索聊天记录、下载聊天中的文件、查看群成员、搜索群、创建群聊或话题群、管理标记数据、管理 Feed 置顶（添加/移除/查询置顶会话）、管理标签数据、处理卡片回调时使用。