MCPInstall

clipify-video-clip-generator

by reason-machinesMedia & Creative

Turn long videos into social-ready clips with auto-detection, face-tracking reframe, and opus-style captions

Installs
228
Last 24h
—
Trending rank
—
Security
Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/reason-machines/devtools-skills --skill clipify-video-clip-generator
  1. 01

    Install

    Run the command in your project. The CLI asks which agents to install for.

  2. 02

    Describe the task

    Ask your agent for something this skill covers. It loads clipify-video-clip-generator automatically when relevant.

  3. 03

    Review the result

    The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

Clipify Video Clip Generator

Skill by ara.so — Devtools Skills collection.

Clipify is a Claude Code skill that automatically turns long-form videos into social-ready clips. It transcribes video, identifies clip-worthy moments, reframes 16:9 to 9:16 with face-tracking pans, and burns opus-style word-by-word captions.

Key capabilities:

  • Auto-detect punchlines, reversals, and awkward pauses via Whisper transcription
  • Face-tracking pan for 9:16 vertical clips (no ML models — uses motion energy)
  • Opus/karaoke/minimal subtitle styles with word-level highlighting
  • Hardware-accelerated rendering (VideoToolbox on macOS)
  • ~20s render time for 20s clips on Apple Silicon

Installation

# Clone to Claude Code skills directory
git clone https://github.com/louisedesadeleer/clipify.git ~/.claude/skills/clipify

# Install dependencies
brew install ffmpeg
pip install openai-whisper numpy

Requirements:

  • macOS (or Linux/Windows with -hwaccel videotoolbox removed from SKILL.md)
  • ffmpeg with libx264
  • Python 3 with numpy
  • Whisper (openai-whisper)

Restart Claude Code after installation. The /clipify slash command will be available.

Usage Workflow

1. Invoke the skill

In Claude Code:

/clipify

Provide the path to your source video when prompted:

/path/to/long-interview.mp4

2. Review proposed clips

Clipify transcribes the video and proposes 3-5 candidates with:

  • Timestamp range
  • Title/description
  • Reason (punchline, reversal, audio peak, awkward pause)

Example output:

Clip 1: "The worst product advice" (02:34 - 02:51)
  Reason: Reversal after awkward pause

Clip 2: "We burned $2M on this" (08:12 - 08:29)
  Reason: Audio peak + punchline

Clip 3: "My co-founder quit on Zoom" (15:03 - 15:24)
  Reason: Punchline

3. Select clip and format

Choose which clip to cut, then specify:

  • Aspect ratio: 9:16 (vertical), 16:9 (horizontal), 1:1 (square)
  • Reframe style (if 9:16 from 16:9 with two speakers): pan (follow speaker) or split-screen
  • Subtitle style: opus (bold white + yellow highlight), karaoke (word-by-word), minimal, or paste reference image

4. Output

Final clips are saved to:

<source-video-dir>/clipify_out/clip_<timestamp>.mp4

Scripts Reference

Clipify uses standalone Python scripts for each processing step. You can call these directly for custom workflows.

analyze.py — Speaker timeline from motion energy

# Generate motion energy files for two face regions
ffmpeg -i video.mp4 -vf "crop=300:200:100:50,format=gray,tblend=all_mode=difference" \
  -f rawvideo -pix_fmt gray - | python scripts/analyze.py motion_left.bin

ffmpeg -i video.mp4 -vf "crop=300:200:1000:50,format=gray,tblend=all_mode=difference" \
  -f rawvideo -pix_fmt gray - | python scripts/analyze.py motion_right.bin

# Analyze both to generate speaker timeline
python scripts/analyze.py motion_left.bin motion_right.bin --fps 30 > timeline.txt

Output format (timeline.txt):

0.00-2.34:left
2.34-5.67:right
5.67-8.12:left

build_pan.py — Generate ffmpeg crop expression

# From speaker timeline, build hard-cut pan expression
python scripts/build_pan.py timeline.txt --left-x 100 --right-x 1000 --width 608 > pan_expr.txt

# Use in ffmpeg crop filter
ffmpeg -i source.mp4 -vf "crop=608:1080:'$(cat pan_expr.txt)':0" output.mp4

Arguments:

  • --left-x: X coordinate of left speaker's face center
  • --right-x: X coordinate of right speaker's face center
  • --width: Width of the 9:16 crop window (e.g., 608 for 1080p)

Output: ffmpeg expression string like:

if(between(t,0,2.34),100,if(between(t,2.34,5.67),1000,if(between(t,5.67,8.12),100,1000)))

build_ass.py — Generate ASS subtitle file

# From Whisper JSON output, create opus-style captions
python scripts/build_ass.py whisper_output.json --style opus > captions.ass

# Burn into video
ffmpeg -i video.mp4 -vf "ass=captions.ass" output.mp4

Whisper JSON format (input):

{
  "segments": [
    {
      "start": 0.5,
      "end": 2.3,
      "text": "This is the worst advice",
      "words": [
        {"word": "This", "start": 0.5, "end": 0.7},
        {"word": "is", "start": 0.7, "end": 0.85},
        {"word": "the", "start": 0.85, "end": 1.0},
        {"word": "worst", "start": 1.0, "end": 1.4},
        {"word": "advice", "start": 1.4, "end": 2.3}
      ]
    }
  ]
}

Styles:

  • opus: Bold white text, yellow active-word highlight, centered top
  • karaoke: Word-by-word color change, bottom positioned
  • minimal: Clean white text, no highlights

audio_align.py — Find clip offset in source

# Find where a 20s clip appears in a 2-hour source video
python scripts/audio_align.py source.mp4 clip.mp4

# Output: 00:15:34.2 (offset timestamp)

Uses audio cross-correlation. Useful for re-linking edited clips to source timestamps.

Common Patterns

Extract clip manually (without auto-detection)

# 1. Transcribe with Whisper
whisper source.mp4 --model base --output_format json --output_dir ./

# 2. Cut segment (03:15 to 03:42)
ffmpeg -i source.mp4 -ss 00:03:15 -to 00:03:42 -c copy raw_clip.mp4

# 3. Generate captions for this segment
python scripts/build_ass.py source.json --start 195 --end 222 --style opus > clip.ass

# 4. Reframe to 9:16 with center crop (no pan)
ffmpeg -i raw_clip.mp4 -vf "crop=608:1080:656:0,ass=clip.ass" final_clip.mp4

Two-speaker pan with manual face coordinates

# 1. Identify face regions on a sample frame
ffplay -ss 00:01:00 source.mp4  # visual inspection

# Left face: x=200, y=100, width=300, height=200
# Right face: x=1100, y=100, width=300, height=200

# 2. Generate motion energy files
ffmpeg -i source.mp4 -ss 00:03:15 -to 00:03:42 \
  -vf "crop=300:200:200:100,format=gray,tblend=all_mode=difference" \
  -f rawvideo -pix_fmt gray motion_left.bin

ffmpeg -i source.mp4 -ss 00:03:15 -to 00:03:42 \
  -vf "crop=300:200:1100:100,format=gray,tblend=all_mode=difference" \
  -f rawvideo -pix_fmt gray motion_right.bin

# 3. Analyze speaker timeline
python scripts/analyze.py motion_left.bin motion_right.bin --fps 30 > timeline.txt

# 4. Build pan expression (faces centered at x=350 and x=1250)
python scripts/build_pan.py timeline.txt --left-x 350 --right-x 1250 --width 608 > pan.txt

# 5. Apply crop with pan
ffmpeg -i source.mp4 -ss 00:03:15 -to 00:03:42 \
  -vf "crop=608:1080:'$(cat pan.txt)':0" panned_clip.mp4

Custom subtitle styling

Edit ASS file generated by build_ass.py:

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Impact,68,&H00FFFFFF,&H0000FFFF,&H00000000,&H00000000,-1,0,0,0,100,100,0,0,1,3,0,2,10,10,120,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.50,0:00:02.30,Default,,0,0,0,,{\k70}This {\k15}is {\k15}the {\k40}worst {\k90}advice

Customize:

  • Fontname: Impact, Arial, Montserrat
  • Fontsize: 68 for 1080p vertical
  • PrimaryColour: &H00FFFFFF (white in BGR hex)
  • SecondaryColour: &H0000FFFF (yellow highlight)
  • Outline: Border thickness (3 = thick black outline)
  • Alignment: 2=bottom center, 8=top center
  • MarginV: Vertical margin from edge

Batch processing multiple clips

import subprocess
import json

# Load Whisper transcript
with open("source.json") as f:
    data = json.load(f)

# Define clip ranges
clips = [
    {"start": 154, "end": 171, "title": "clip1"},
    {"start": 492, "end": 509, "title": "clip2"},
    {"start": 903, "end": 924, "title": "clip3"}
]

for clip in clips:
    # Cut raw clip
    subprocess.run([
        "ffmpeg", "-i", "source.mp4",
        "-ss", str(clip["start"]),
        "-to", str(clip["end"]),
        "-c", "copy",
        f"raw_{clip['title']}.mp4"
    ])
    
    # Generate captions
    subprocess.run([
        "python", "scripts/build_ass.py", "source.json",
        "--start", str(clip["start"]),
        "--end", str(clip["end"]),
        "--style", "opus"
    ], stdout=open(f"{clip['title']}.ass", "w"))
    
    # Reframe and burn captions
    subprocess.run([
        "ffmpeg", "-i", f"raw_{clip['title']}.mp4",
        "-vf", f"crop=608:1080:656:0,ass={clip['title']}.ass",
        f"{clip['title']}_final.mp4"
    ])

Configuration

Hardware acceleration

macOS (default):

ffmpeg -hwaccel videotoolbox -i input.mp4 ...

Linux with NVIDIA:

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 ...

Windows:

ffmpeg -hwaccel dxva2 -i input.mp4 ...

Disable (CPU only): Remove -hwaccel flags from SKILL.md ffmpeg commands.

Whisper model size

Faster but less accurate:

whisper video.mp4 --model tiny  # ~1GB, 10x faster

More accurate but slower:

whisper video.mp4 --model medium  # ~1.5GB, 2x slower
whisper video.mp4 --model large   # ~3GB, 4x slower

Default in SKILL.md: base (good balance for dialogue).

Output quality settings

High quality (larger file):

ffmpeg -i input.mp4 -c:v libx264 -preset slow -crf 18 -c:a aac -b:a 192k output.mp4

Fast encode (lower quality):

ffmpeg -i input.mp4 -c:v libx264 -preset veryfast -crf 23 -c:a aac -b:a 128k output.mp4

Social media optimized (SKILL.md default):

ffmpeg -i input.mp4 -c:v libx264 -preset medium -crf 20 -c:a aac -b:a 160k output.mp4

Troubleshooting

"No motion detected in face regions"

Face crop coordinates are wrong. Verify on a sample frame:

# Extract frame at 1 minute mark
ffmpeg -ss 00:01:00 -i source.mp4 -frames:v 1 sample.png

# Overlay crop rectangles (adjust x,y,w,h)
ffmpeg -i source.mp4 -ss 00:01:00 -frames:v 1 \
  -vf "drawbox=x=200:y=100:w=300:h=200:color=red:t=5,drawbox=x=1100:y=100:w=300:h=200:color=blue:t=5" \
  sample_boxes.png

Red = left face, blue = right face. Adjust coordinates until boxes frame each person's mouth/chin.

"Captions out of sync"

Whisper timestamps drift on long videos. Use smaller segments:

# Transcribe only the relevant 5-minute section
ffmpeg -i source.mp4 -ss 00:15:00 -to 00:20:00 -c copy segment.mp4
whisper segment.mp4 --model base --output_format json

Or enable Whisper's word-level timestamps:

whisper video.mp4 --model base --word_timestamps True

"Pan cuts too frequently"

Increase motion threshold in analyze.py:

# Default threshold
MOTION_THRESHOLD = 0.1

# Higher = less sensitive (fewer speaker changes)
MOTION_THRESHOLD = 0.3

Or add minimum duration between cuts in build_pan.py:

# Ignore speaker changes shorter than 2 seconds
MIN_SEGMENT_DURATION = 2.0

"ffmpeg not found"

Ensure ffmpeg is in PATH:

which ffmpeg
# Should print: /usr/local/bin/ffmpeg or similar

# If not installed:
brew install ffmpeg  # macOS
sudo apt install ffmpeg  # Ubuntu

"Whisper import error"

# Uninstall conflicting whisper packages
pip uninstall whisper openai-whisper

# Reinstall correct package
pip install openai-whisper

# Verify
python -c "import whisper; print(whisper.__version__)"

"VideoToolbox acceleration failed"

macOS-specific. Fallback to CPU:

# Remove -hwaccel videotoolbox from all ffmpeg commands
ffmpeg -i input.mp4 ...  # (no -hwaccel flag)

Or use software decode explicitly:

ffmpeg -hwaccel none -i input.mp4 ...

Advanced: Integration with Custom Workflows

Use Clipify detection with external editor

# 1. Run Clipify's detection logic (without cutting)
# This would be in your custom script:

import subprocess
import json

# Transcribe
subprocess.run(["whisper", "source.mp4", "--model", "base", "--output_format", "json"])

# Load transcript
with open("source.json") as f:
    transcript = json.load(f)

# Simple punchline detector (look for laughter indicators)
candidates = []
for segment in transcript["segments"]:
    text = segment["text"].lower()
    if any(word in text for word in ["haha", "lol", "crazy", "worst", "insane"]):
        candidates.append({
            "start": segment["start"],
            "end": segment["end"],
            "text": segment["text"]
        })

print(json.dumps(candidates, indent=2))
# Output to DaVinci Resolve, Premiere, etc.

Export timeline for manual editing

# Generate EDL (Edit Decision List) from Clipify candidates
def to_edl(clips, fps=30):
    edl = ["TITLE: Clipify Export", "FCM: NON-DROP FRAME", ""]
    
    for i, clip in enumerate(clips, 1):
        start_tc = frames_to_tc(int(clip["start"] * fps), fps)
        end_tc = frames_to_tc(int(clip["end"] * fps), fps)
        
        edl.append(f"{i:03d}  AX       V     C        {start_tc} {end_tc} 00:00:00:00 {end_tc}")
    
    return "\n".join(edl)

def frames_to_tc(frames, fps):
    h = frames // (fps * 3600)
    m = (frames % (fps * 3600)) // (fps * 60)
    s = (frames % (fps * 60)) // fps
    f = frames % fps
    return f"{h:02d}:{m:02d}:{s:02d}:{f:02d}"

# Usage
clips = [{"start": 154.2, "end": 171.8}, {"start": 492.5, "end": 509.1}]
print(to_edl(clips))

Custom caption animations

Modify ASS file for animated entrances:

Dialogue: 0,0:00:00.50,0:00:02.30,Default,,0,0,0,,{\fad(200,200)\move(640,1000,640,900)}This is animated text
  • \fad(200,200): 200ms fade in/out
  • \move(x1,y1,x2,y2): Slide from bottom to center
  • \t(0,500,\fscx120\fscy120): Scale animation over 500ms

Performance Tips

  1. Use proxy files for preview: Transcode to lower resolution before clipify analysis

    ffmpeg -i source.mp4 -vf scale=960:540 -c:v libx264 -crf 28 proxy.mp4

  2. Skip transcription on re-runs: Cache Whisper JSON output

    if [ ! -f source.json ]; then
  whisper source.mp4 --model base --output_format json
fi

  3. Parallel clip rendering: Process multiple clips simultaneously

    for clip in clip1 clip2 clip3; do
  ffmpeg -i $clip.mp4 -vf "..." ${clip}_out.mp4 &
done
wait

  4. GPU acceleration: Use NVIDIA NVENC for faster encoding

    ffmpeg -hwaccel cuda -i input.mp4 -c:v h264_nvenc -preset p4 -crf 20 output.mp4

FAQ

What does the clipify-video-clip-generator skill do?

Turn long videos into social-ready clips with auto-detection, face-tracking reframe, and opus-style captions

How do I install clipify-video-clip-generator?

Run npx skills add https://github.com/reason-machines/devtools-skills --skill clipify-video-clip-generator in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/clipify-video-clip-generator/ (personal) or .claude/skills/clipify-video-clip-generator/ (project).

Which AI agents can use clipify-video-clip-generator?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is clipify-video-clip-generator safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains clipify-video-clip-generator?

It is published by reason-machines in reason-machines/devtools-skills. It has 228 recorded installs.

More from reason-machines/devtools-skills

View all →

cc-switch-cli

reason-machines/devtools-skills

Cross-platform CLI tool for managing Claude Code, Codex, Gemini, OpenCode & OpenClaw providers, MCP servers, prompts, skills, and proxies.

AI & Agents316 installs

wx-cli-wechat-local-data

reason-machines/devtools-skills

Open the skill to read its SKILL.md instructions.

284 installs

chrome-devtools-mcp-automation

reason-machines/devtools-skills

Use Chrome DevTools MCP to control Chrome, debug web apps, analyze performance, and automate browser tasks via MCP tools

AI & Agents265 installs

officecli-office-automation

reason-machines/devtools-skills

Use OfficeCLI to create, read, and edit Word, Excel, and PowerPoint files from the command line with AI-friendly commands.

Productivity & Automation263 installs

cli-anything-agent-native-software

reason-machines/devtools-skills

Generate and use command-line interfaces that make any software controllable by AI agents with structured JSON output

AI & Agents258 installs

autocli-web-scraping

reason-machines/devtools-skills

Open the skill to read its SKILL.md instructions.

Productivity & Automation257 installs

Popular media & creative skills

View category →

hyperframes

heygen-com/hyperframes

Read first to make, edit, animate or render any video with HyperFrames (HTML rendered to video), including launch films, explainers, music videos, charts, maps, vertical clips, title cards, captions and overlays. Holds the render contract, the work loop and what each kind of video needs.

Media & Creative1.2M installs

media-use

heygen-com/hyperframes

Get or make any media for a HyperFrames video - music, sound effects, images, icons, logos, voiceover, avatar and generated video, transcripts, cutouts, colour grades and looks - with HeyGen or free local tools, and mix placed audio. Also for footage that looks dark, flat or dull, or should be stylized or hidden.

Media & Creative1.1M installs

hyperframes-registry

heygen-com/hyperframes

About 450 ready-made HyperFrames blocks and components installed as editable source - transitions, overlays, titles, kinetic type, captions, charts, map overlays, 3D, camera moves. Use it before hand-building any named visual (CRT scanlines, glitch, film grain, a chart); also for hyperframes add and catalog.

Media & Creative1M installs

hyperframes-creative

heygen-com/hyperframes

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use hyperframes-animation.

Media & Creative916.5K installs

ai-video-generation

101-skills/superpowers

Generate AI videos with Google Veo, Seedance 2.0, HappyHorse, Wan, Grok and 40+ models via inference.sh CLI. Models: Veo 3.1, Veo 3, Seedance 2.0, HappyHorse 1.0, Wan 2.5, Grok Imagine Video, OmniHuman, Fabric, HunyuanVideo. Capabilities: text-to-video, image-to-video, reference-to-video, video editing, lipsync, avatar animation, video upscaling, foley sound. Use for: social media videos, marketing content, explainer videos, product demos, AI avatars. Triggers: video generation, ai video, text to video, image to video, veo, animate image, video from image, ai animation, video generator, generate video, t2v, i2v, ai video maker, create video with ai, runway alternative, pika alternative, sora alternative, kling alternative, seedance, happyhorse

Media & Creative881.5K installs

ai-image-generation

101-skills/superpowers

Generate AI images with GPT-Image-2, FLUX, Gemini, Grok, Seedream, Reve and 50+ models via inference.sh CLI. Models: GPT-Image-2, FLUX Dev LoRA, FLUX.2 Klein LoRA, Gemini 3 Pro Image, Grok Imagine, Seedream 4.5, Reve, ImagineArt. Capabilities: text-to-image, image-to-image, inpainting, LoRA, image editing, upscaling, text rendering. Use for: AI art, product mockups, concept art, social media graphics, marketing visuals, illustrations. Triggers: flux, image generation, ai image, text to image, stable diffusion, generate image, ai art, midjourney alternative, dall-e alternative, text2img, t2i, image generator, ai picture, create image with ai, generative ai, ai illustration, grok image, gemini image, gpt image, openai image, chatgpt image

Media & Creative880.3K installs