grepai-chunking

Q: How do I install grepai-chunking?

Run npx skills add https://github.com/yoanbernabeu/grepai-skills --skill grepai-chunking in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/grepai-chunking/ (personal) or .claude/skills/grepai-chunking/ (project).

by yoanbernabeuAI & Agents

Configure code chunking in GrepAI. Use this skill to optimize how code is split for embedding.

Installs: 703
Last 24h: —
Trending rank: —
Security: Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/yoanbernabeu/grepai-skills --skill grepai-chunking

01
Install
Run the command in your project. The CLI asks which agents to install for.
02
Describe the task
Ask your agent for something this skill covers. It loads grepai-chunking automatically when relevant.
03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

GrepAI Chunking Configuration

This skill covers how GrepAI splits code files into chunks for embedding, and how to optimize chunking for your codebase.

When to Use This Skill

Optimizing search accuracy
Adjusting for code style (verbose vs. concise)
Troubleshooting search results
Understanding how indexing works

What is Chunking?

Chunking is the process of splitting source files into smaller segments for embedding:

┌─────────────────────────────────────┐
│         Large Source File           │
│         (1000+ tokens)              │
└─────────────────────────────────────┘
                  ↓
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Chunk 1 │ │ Chunk 2 │ │ Chunk 3 │
│ ~512    │ │ ~512    │ │ ~512    │
│ tokens  │ │ tokens  │ │ tokens  │
└─────────┘ └─────────┘ └─────────┘
                  ↓
          Each chunk gets
          its own embedding

Why Chunking Matters

Embedding models have optimal input sizes:

Too large chunks: Less precise search results
Too small chunks: Lost context, fragmented results
Just right: Good balance of precision and context

Configuration

Basic Settings

# .grepai/config.yaml
chunking:
  size: 512      # Tokens per chunk
  overlap: 50    # Overlap between chunks

Understanding Parameters

Chunk Size

The target number of tokens per chunk.

Size	Effect
256	More precise, less context
512	Balanced (default)
1024	More context, less precise

Overlap

Tokens shared between adjacent chunks. Preserves context at boundaries.

Overlap	Effect
0	No overlap, may lose context at boundaries
50	Standard overlap (default)
100	More context, larger index

Visualization

With size=512 and overlap=50:

File: auth.go (1000 tokens)

Chunk 1: tokens 1-512
         ┌────────────────────────────────────┐
         │ func Login(user, pass)...          │
         └────────────────────────────────────┘
                                    ↘
                              50 token overlap
                                    ↙
Chunk 2: tokens 463-974
         ┌────────────────────────────────────┐
         │ ...validate credentials...         │
         └────────────────────────────────────┘
                                    ↘
                              50 token overlap
                                    ↙
Chunk 3: tokens 925-1000
         ┌──────────────┐
         │ ...return    │
         └──────────────┘

Recommended Settings by Language

Verbose Languages (Java, C#)

chunking:
  size: 768    # Larger to capture full methods
  overlap: 75

Concise Languages (Go, Python)

chunking:
  size: 512    # Standard size
  overlap: 50

Very Concise (Rust, Zig)

chunking:
  size: 384    # Smaller for precise results
  overlap: 40

Recommended Settings by Codebase

Small Functions (Microservices)

chunking:
  size: 384    # Capture individual functions
  overlap: 40

Large Classes (Monolith)

chunking:
  size: 768    # Capture more context
  overlap: 100

Mixed Codebase

chunking:
  size: 512    # Balanced default
  overlap: 50

How Tokens are Counted

GrepAI uses approximate token counting:

~4 characters = 1 token (for English text)
Code varies based on identifiers and syntax

Example:

func calculateTotal(items []Item) float64 {
    total := 0.0
    for _, item := range items {
        total += item.Price * float64(item.Quantity)
    }
    return total
}

≈ 45 tokens

Impact on Index Size

Larger overlap = more chunks = larger index:

Size	Overlap	Chunks per 10K tokens	Index Impact
512	0	~20	Smallest
512	50	~22	Standard
512	100	~24	+10%
256	50	~44	+100%

Impact on Search Quality

Too Small Chunks (size: 128)

Query: "authentication middleware"

Result: "...c.AbortWithStatus(401)..."
        (Fragment, missing context)

Just Right (size: 512)

Query: "authentication middleware"

Result: "func AuthMiddleware() gin.HandlerFunc {
            return func(c *gin.Context) {
                token := c.GetHeader("Authorization")
                if token == "" {
                    c.AbortWithStatus(401)
                    return
                }
                // validate token...
            }
        }"
        (Complete function with context)

Too Large Chunks (size: 2048)

Query: "authentication middleware"

Result: "// Multiple unrelated functions...
        func AuthMiddleware()... (your match)
        func LoggingMiddleware()...
        func CORSMiddleware()..."
        (Too much noise)

Experimentation

Testing Different Settings

Try smaller chunks for more precise results:

chunking:
  size: 384
  overlap: 40

Re-index:

rm .grepai/index.gob
grepai watch

Test with searches:

grepai search "your query"

Adjust and repeat until satisfied.

Comparing Results

Before changing settings, save a search result:

grepai search "authentication" > before.txt

After changing settings and re-indexing:

grepai search "authentication" > after.txt
diff before.txt after.txt

Chunk Boundaries

GrepAI tries to split at logical boundaries:

Empty lines (function/class boundaries)
Closing braces
Statement ends

This means actual chunk sizes may vary slightly from the target.

Best Practices

Start with defaults: 512/50 works well for most codebases
Adjust based on code style: Verbose = larger, concise = smaller
Test with real queries: See what your searches return
Re-index after changes: Must regenerate embeddings
Consider overlap: Don't set to 0 unless index size is critical

Common Issues

❌ Problem: Search results are too fragmented ✅ Solution: Increase chunk size:

chunking:
  size: 768

❌ Problem: Search results have too much irrelevant context ✅ Solution: Decrease chunk size:

chunking:
  size: 384

❌ Problem: Results miss related code at function boundaries ✅ Solution: Increase overlap:

chunking:
  overlap: 100

❌ Problem: Index is too large ✅ Solutions:

Decrease overlap
Increase chunk size
Add more ignore patterns

Output Format

Chunking status:

✅ Chunking Configuration

   Size: 512 tokens
   Overlap: 50 tokens

   Index Statistics:
   - Total files: 245
   - Total chunks: 1,234
   - Avg chunks/file: 5.0
   - Avg chunk size: 478 tokens

   Recommendations:
   - Current settings are balanced
   - Consider size: 384 for more precise results
   - Consider size: 768 for more context

FAQ

What does the grepai-chunking skill do?

Configure code chunking in GrepAI. Use this skill to optimize how code is split for embedding.

How do I install grepai-chunking?

Run npx skills add https://github.com/yoanbernabeu/grepai-skills --skill grepai-chunking in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/grepai-chunking/ (personal) or .claude/skills/grepai-chunking/ (project).

Which AI agents can use grepai-chunking?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is grepai-chunking safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains grepai-chunking?

It is published by yoanbernabeu in yoanbernabeu/grepai-skills. It has 703 recorded installs.

Popular ai & agents skills

View category →

google-agents-cli-publish

google/agents-cli

This skill should be used when the user wants to "publish an agent", "publish my ADK agent", "register an agent with Gemini", "publish to Gemini" (formerly Gemini Enterprise), or needs guidance on the agents-cli publish gemini command. Also use when the user wants to "manage agents in Agent Registry", "list/update/delete registered agents", or "register an MCP server". Covers ADK vs A2A registration modes, programmatic and interactive usage, flag reference, auto-detection from deployment metadata, Agent Registry fleet management, and troubleshooting. Part of the agents-cli skills suite. Do NOT use for deployment (use google-agents-cli-deploy).

AI & Agents513.9K installs

google-agents-cli-eval

google/agents-cli

This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).

AI & Agents513.5K installs

skill-creator

anthropics/skills

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

OfficialAI & Agents416.4K installs

codex-pet

prime-skills/runcomfy-agent-skills

Codex Pet generator on RunComfy. Build a Codex-compatible Codex Pet spritesheet.webp + pet.json from a single reference image, drop it into ${CODEXHOME:-$HOME/.codex}/pets/ / and Codex picks it up as a custom Codex Pet next to the 8 built-ins. This skill produces the exact Codex Pet atlas Codex expects (1536x1872 PNG/WebP, 8 cols x 9 rows, 192x208 cells, 9 animation states — idle, running-right, running-left, waving, jumping, failed, waiting, running, review). Calls OpenAI GPT Image 2 edit ONCE via the local RunComfy CLI as runcomfy run openai/gpt-image-2/edit to produce a canonical Codex Pet pose, then assembles all 9 animation rows programmatically with ImageMagick micro-transforms — no Codex Pro, no $imagegen, no OPENAIAPIKEY required, only RUNCOMFYTOKEN. Triggers on "codex pet", "create codex pet", "make codex pet", "hatch codex pet", "/hatch image", "desktop pet codex", "codex pets", "spritesheet.webp", or any explicit ask to build a custom pet for OpenAI Codex.

AI & Agents402.1K installs

full-output-enforcement

leonxlnx/taste-skill

Overrides default LLM truncation behavior. Enforces complete code generation, bans placeholder patterns, and handles token-limit splits cleanly. Apply to any task requiring exhaustive, unabridged output.

AI & Agents401K installs

executing-plans

obra/superpowers

Use when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available

AI & Agents245.7K installs

grepai-chunking

Install

Install

Describe the task

Review the result

SKILL.md

GrepAI Chunking Configuration

When to Use This Skill

What is Chunking?

Why Chunking Matters

Configuration

Basic Settings

Understanding Parameters

Chunk Size

Overlap

Visualization

Recommended Settings by Language

Verbose Languages (Java, C#)

Concise Languages (Go, Python)

Very Concise (Rust, Zig)

Recommended Settings by Codebase

Small Functions (Microservices)

Large Classes (Monolith)

Mixed Codebase

How Tokens are Counted

Impact on Index Size

Impact on Search Quality

Too Small Chunks (size: 128)

Just Right (size: 512)

Too Large Chunks (size: 2048)

Experimentation

Testing Different Settings

Comparing Results

Chunk Boundaries

Best Practices

Common Issues

Output Format

FAQ

More from yoanbernabeu/grepai-skills

Popular ai & agents skills