MCPInstall

game-automation

by rehan-remadeMedia & Creative

Launch, see and drive a real game so an agent can test its own mods. Covers launching via Steam or the exe, windowed mode at a fixed size, screenshots of the game window (GPU-safe capture), clicking and typing into menus, holding keys, relative mouse, crash-reporter cleanup, killing by PID, reading logs, scripted test scenes, and an in-game JSON bridge that lets an agent observe and act. Windows-first (also from WSL), with notes for Linux and macOS. Use when a mod must be verified in game, when navigating menus automatically, or when building a repeatable test or demo scene.

Installs
1.5K
Last 24h
+21
Trending rank
#5736
Security
Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/rehan-remade/universal-modder --skill game-automation
  1. 01

    Install

    Run the command in your project. The CLI asks which agents to install for.

  2. 02

    Describe the task

    Ask your agent for something this skill covers. It loads game-automation automatically when relevant.

  3. 03

    Review the result

    The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

Game automation: the agent's hands and eyes

The real game is the oracle. The loop is launch → navigate → set up the scene → act → screenshot and read logs → decide. Build it once per game, then every change you make can be checked in a minute.

Windows (native or from WSL): um win

um win setup                                   # once: PowerShell tools + an ffmpeg with gfxcapture, picks NVENC/AMF/QSV/x264
um win ps                                      # windowed processes: pid, name, title
um win launch --steam 105600                   # or: um win launch "C:\Games\Foo\Foo.exe" -- -windowed
um win shot --exe Terraria.exe shot.png --scale 0.33   # full frame + a 1/3 copy that's cheap to look at
um win drive --proc Terraria "focus" "click 640 360" "key 0x1B" "type hello" "hold 0x44 1500"
um win drive --proc Terraria "hold 0x20 150" "sleep 200" "hold 0x20 150"   # sleep <ms> pauses between steps
um win drive --proc Terraria idle              # seconds since the user last touched mouse/keyboard (does not wait)
um win kill <pid>                              # exact PID only
um win reg get "HKCU\Software\..."             # registry (reg set backs the key up first)

From Python (for longer scripts), from um.win import Drive, shot, Recorder, then d = Drive("AoE2DE_s"), d.focus(), d.click(x, y), d.key("0x0D").

WinDrive commands (um/ps1/WinDrive.ps1; stdin protocol, one reply per line). Coordinates are in the game window's client area.

  • Mouse: move x y, click x y [right], mdown/mup, drag x0 y0 x1 y1, rel dx dy (FPS cameras / raw input), wheel 120.
  • Keyboard: key <vk> [tap|down|up], hold <vk> <ms>, type <text>, scanmode on (DirectInput / raw-input games that ignore virtual-key events).
  • Window: focus, rect, size <w> <h> (sets the client size), title, fg, idle, untop.
  • Timing: sleep <ms> waits between steps in one call. Each um win drive call starts PowerShell (1-2 s), so put timed combos in one call.
  • Virtual-key codes: Esc 0x1B, Enter 0x0D, Space 0x20, W/A/S/D 0x57/0x41/0x53/0x44, F1 0x70, Shift 0x10, Ctrl 0x11, arrows 0x25-0x28.

Rules of the road

  • Never focus a game with an online mode while the human is typing. It happened with GTA V: the agent focused GTA to click Story Mode, the human's keystrokes hit GTA's landing page, and GTA warned about "accessing GTA Online servers with an altered version". ScriptHookV blocked it; don't rely on that. Check idle, ask the human to click, and launch with the anti-cheat off so online can't start. See knowledge/techniques/driving-real-games-safely.md.
  • Input only goes to the game. WinDrive refuses to send while another app is in the foreground. The one exception is when nothing is and the cursor is over the game, which windowed games cause by dropping the foreground on clicks.
  • The user may be at the PC. If idle is under a minute, ask before driving, and keep sessions short. Unattended runs are fine once the user says so.
  • Screenshots cost tokens. Look at --scale 0.33 copies. Multiply coordinates back ×3 for clicks, and keep a table of menu click points in MODLOG.md, measured once.
  • Why gfxcapture: GPU-rendered games come out black with GDI capture. gfxcapture (Windows.Graphics.Capture) grabs one window's real frames, even when covered. It can't capture minimized windows.
  • HDR displays: with Windows Auto HDR on for an SDR game, captures come out washed out (2-3x brighter, cyan-shifted), so colours can't be judged. um win shot/record warn when it's on; turn it off for the game while capturing (Settings > System > Display > Graphics > the game > Auto HDR), and back on after if the user wants.

Make the window predictable

  • Windowed mode at a fixed client size. Every game hides this setting somewhere:
    • an ini ([/Script/Engine.GameUserSettings] in UE GameUserSettings.ini, Unity's Screenmanager Fullscreen mode / Resolution registry values under HKCU\Software\<company>\<product>);
    • a launch flag (-windowed -w 1920 -h 1080, Unity -screen-fullscreen 0 -screen-width 1920 -screen-height 1080, Source -sw -w 1920 -h 1080);
    • or the game's own registry (AoE2: Mode Display 0 + Windowed Width/Height).
    • um win drive --proc X "size 1920 1080" also resizes most windowed games.
  • Skip intros: launch flags (-skipintro, -nosplash, Unity -popupwindow), or delete/rename intro videos in a copy of the game.
  • Jump straight in: loader flags (tModLoader -skipselect Player:World), a save made for testing, scenario auto-load, a dev console command.
  • Background throttling: many games throttle or pause unfocused. Find the setting (Terraria: Main.instance.InactiveSleepTime = TimeSpan.Zero; Unity: Application.runInBackground = true; UE: t.IdleWhenNotForeground 0).

Better than clicking: a scripted scene or a bridge

  • Chat or console commands in your mod give items, spawn enemies and teleport. They make tests one line (/arsenal, /mothership).
  • Test scenes: a mod-side timeline (spawn waves at frame N, fire at N+30), a scenario with triggers (AoE2 via AoE2ScenarioParser), a dedicated test world with a pristine backup.
  • Agent bridge: a JSON-lines socket inside the mod on 127.0.0.1 (see examples/terraria-tmodloader/reference/AgentBridge.cs).
    • Commands like observe (menu options or world state as text), click <id>, controls, step.
    • Handle every request on the game's main thread after an update, so replies are consistent.
    • Read menus generically from the UI tree, so the agent sees what a player would. Leave destructive buttons (delete) out.
    • An agent can then play without pixels at all.
  • Out-of-process backend: keep the in-game part thin and put the brains outside (HTTP/file-drop). That pattern scales to AI NPCs and cross-game mashups.

Crashes and cleanup

  • Crash reporters (BugSplat BsSndRpt64.exe, CrashReportClient.exe, UnityCrashHandler64.exe) can linger and make Steam say "already running". um win ps to spot them, um win kill <pid> to clear them.
  • Never pkill -f or wildcard taskkill /IM. The pattern can match your own shell or other apps.
  • After a crash, read the loader or game log first (the mod-any-game skill lists them), then the Windows Event Viewer (Application log) for native crashes.

Linux and macOS

  • Linux (X11): xdotool search --name "Game" windowactivate --sync key Escape, xdotool mousemove --window $W 640 360 click 1. Screenshots: import -window $(xdotool search --name Game) shot.png, or ffmpeg x11grab.
  • Linux (Wayland): ydotool + grim. Proton games are Windows games under Wine: launch options (WINEDLLOVERRIDES, PROTON_LOG=1) go in Steam.
  • macOS: screencapture -l <windowid> shot.png (window id via GetWindowID or AppleScript), osascript / cliclick for input. Input and screen recording need Accessibility and Screen Recording permission for the terminal.

FAQ

What does the game-automation skill do?

Launch, see and drive a real game so an agent can test its own mods. Covers launching via Steam or the exe, windowed mode at a fixed size, screenshots of the game window (GPU-safe capture), clicking and typing into menus, holding keys, relative mouse, crash-reporter cleanup, killing by PID, reading logs, scripted test scenes, and an in-game JSON bridge that lets an agent observe and act. Windows-first (also from WSL), with notes for Linux and macOS. Use when a mod must be verified in game, when navigating menus automatically, or when building a repeatable test or demo scene.

How do I install game-automation?

Run npx skills add https://github.com/rehan-remade/universal-modder --skill game-automation in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/game-automation/ (personal) or .claude/skills/game-automation/ (project).

Which AI agents can use game-automation?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is game-automation safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains game-automation?

It is published by rehan-remade in rehan-remade/universal-modder. It has 1,523 recorded installs.

More from rehan-remade/universal-modder

View all →

mod-any-game

rehan-remade/universal-modder

Mod a PC game the user owns, taking an idea to working in the real game and recorded. It covers new items, weapons, enemies, bosses, units, civilizations, mechanics, art, sound, VR and cross-game mashups. Use when the user wants to mod, extend, hack on, reverse engineer or mash up a game ("add a nuke to Terraria", "make a new civ for Age of Empires", "put Minecraft inside X", "can I mod this game?"). It covers recon (engine, loaders, anti-cheat), choosing the route, a safe lab, a first vertical slice, fal-generated assets, in-game verification and a showcase clip.

Media & Creative1.6K installs

reverse-engineering

rehan-remade/universal-modder

Read how a game actually works so a mod can hook it. Decompile .NET/Mono (ILSpy), IL2CPP (Cpp2IL, Il2CppDumper), Java (Vineflower) and native code (Ghidra, IDA, via MCP servers); dump data and asset formats; reverse-engineer a binary file format and prove it with a round trip; read live memory (Cheat Engine, Frida, x64dbg); find render passes (RenderDoc). Use when a mod needs game internals, e.g. "how does the boss AI work", "find the damage function", "what format is this .sld/.pak/.dat", "where is the player position in memory".

Backend & APIs1.6K installs

game-recon

rehan-remade/universal-modder

Figure out how a specific installed game can be modded, before writing any mod code. Covers where it's installed, the engine and version, managed vs native code, anti-cheat, loaders already present, save and config folders, and the community's modding route. Also whether this machine can run it (a Windows game on macOS or Linux, x64 on ARM). Use when the user asks "can I mod ?", "what engine is ", "how do people mod ", or at the start of any modding task. Produces MODDINGPLAN.md.

Media & Creative1.6K installs

mashup-mods

rehan-remade/universal-modder

Build cross-game mashups and total conversions, the "Minecraft inside Elden Ring" or "skateboarding in MW2" kind. Covers passthrough mods (two games running at once and exchanging state, rendering and collision), porting one game's content or mechanics into another, embedding a decompiled game as a library, and clean reimplementations that read the user's own game files. Use when the user wants to combine two games, bring an enemy, mechanic or world from one game into another, or rebuild a game's runtime, or reimplement the guest's rules headless inside a real host.

Media & Creative1.5K installs

asset-pipeline

rehan-remade/universal-modder

Turn generated or hand-made art into exactly what a game engine loads. Covers sprite cutout, trim and nearest-neighbour fitting to frame sizes, pixelation to a palette, sprite sheets and strips, player/team-colour masks, seamless textures, and rendering a 3D model into sprite frames from the game's own camera (isometric/RTS 8 or 16 headings, side view, top-down) with Blender. Use after generating assets, or when the user asks to make sprites, sprite sheets, animation frames, isometric unit sprites or icons for a game.

Media & Creative1.5K installs

share-field-notes

rehan-remade/universal-modder

Search and contribute to the shared AI-modding knowledge base, where field notes record how specific games were modded, decompiled or reverse-engineered (exact versions, route, engine facts, verification, gotchas). Use before starting work on a game ("has anyone modded X?"), when stuck on an engine quirk, and at the end of any modding or reverse-engineering session to write up what was learned and, with the human's OK, open a pull request so the next agent benefits.

Docs & Writing1.5K installs

Popular media & creative skills

View category →

hyperframes

heygen-com/hyperframes

Read first to make, edit, animate or render any video with HyperFrames (HTML rendered to video), including launch films, explainers, music videos, charts, maps, vertical clips, title cards, captions and overlays. Holds the render contract, the work loop and what each kind of video needs.

Media & Creative1.2M installs

media-use

heygen-com/hyperframes

Get or make any media for a HyperFrames video - music, sound effects, images, icons, logos, voiceover, avatar and generated video, transcripts, cutouts, colour grades and looks - with HeyGen or free local tools, and mix placed audio. Also for footage that looks dark, flat or dull, or should be stylized or hidden.

Media & Creative1.1M installs

hyperframes-registry

heygen-com/hyperframes

About 450 ready-made HyperFrames blocks and components installed as editable source - transitions, overlays, titles, kinetic type, captions, charts, map overlays, 3D, camera moves. Use it before hand-building any named visual (CRT scanlines, glitch, film grain, a chart); also for hyperframes add and catalog.

Media & Creative1M installs

hyperframes-creative

heygen-com/hyperframes

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use hyperframes-animation.

Media & Creative916.5K installs

ai-video-generation

101-skills/superpowers

Generate AI videos with Google Veo, Seedance 2.0, HappyHorse, Wan, Grok and 40+ models via inference.sh CLI. Models: Veo 3.1, Veo 3, Seedance 2.0, HappyHorse 1.0, Wan 2.5, Grok Imagine Video, OmniHuman, Fabric, HunyuanVideo. Capabilities: text-to-video, image-to-video, reference-to-video, video editing, lipsync, avatar animation, video upscaling, foley sound. Use for: social media videos, marketing content, explainer videos, product demos, AI avatars. Triggers: video generation, ai video, text to video, image to video, veo, animate image, video from image, ai animation, video generator, generate video, t2v, i2v, ai video maker, create video with ai, runway alternative, pika alternative, sora alternative, kling alternative, seedance, happyhorse

Media & Creative881.5K installs

ai-image-generation

101-skills/superpowers

Generate AI images with GPT-Image-2, FLUX, Gemini, Grok, Seedream, Reve and 50+ models via inference.sh CLI. Models: GPT-Image-2, FLUX Dev LoRA, FLUX.2 Klein LoRA, Gemini 3 Pro Image, Grok Imagine, Seedream 4.5, Reve, ImagineArt. Capabilities: text-to-image, image-to-image, inpainting, LoRA, image editing, upscaling, text rendering. Use for: AI art, product mockups, concept art, social media graphics, marketing visuals, illustrations. Triggers: flux, image generation, ai image, text to image, stable diffusion, generate image, ai art, midjourney alternative, dall-e alternative, text2img, t2i, image generator, ai picture, create image with ai, generative ai, ai illustration, grok image, gemini image, gpt image, openai image, chatgpt image

Media & Creative880.3K installs