peekaboo
Capture and automate macOS UI with the Peekaboo CLI.
When & Why to Use This Skill
Peekaboo is a comprehensive macOS UI automation CLI designed to bridge the gap between programmatic scripts and the visual interface. It enables developers and AI agents to capture, inspect, and manipulate macOS UI elements with precision. By combining vision-based analysis with direct system interaction, Peekaboo allows for the automation of complex desktop workflows, window management, and input simulation that traditional APIs cannot reach.
Use Cases
- Automated UI Testing: Perform robust regression testing on macOS applications by targeting UI elements via IDs, queries, or visual coordinates.
- Desktop Workflow Scripting: Automate repetitive system tasks such as configuring app settings, managing window layouts, and navigating multi-level menus using JSON-supported commands.
- AI Agent Empowerment: Enable LLM-based agents to 'see' the desktop environment through annotated UI maps and execute human-like interactions like clicking, dragging, and typing.
- Technical Documentation & Monitoring: Programmatically capture high-quality, annotated screenshots or video frames to document software behavior or monitor system states.
- System Integration: Use the CLI to bridge non-scriptable legacy applications into modern automation pipelines via clipboard manipulation and simulated hotkeys.
| name | peekaboo |
|---|---|
| description | Capture and automate macOS UI with the Peekaboo CLI. |
| homepage | https://peekaboo.boo |
| metadata | {"clawdbot":{"emoji":"👀","os":["darwin"],"requires":{"bins":["peekaboo"]},"install":[{"id":"brew","kind":"brew","formula":"steipete/tap/peekaboo","bins":["peekaboo"],"label":"Install Peekaboo (brew)"}]}} |
Peekaboo
Peekaboo is a full macOS UI automation CLI: capture/inspect screens, target UI
elements, drive input, and manage apps/windows/menus. Commands share a snapshot
cache and support --json/-j for scripting. Run peekaboo or
peekaboo <cmd> --help for flags; peekaboo --version prints build metadata.
Tip: run via polter peekaboo to ensure fresh builds.
Features (all CLI capabilities, excluding agent/MCP)
Core
bridge: inspect Peekaboo Bridge host connectivitycapture: live capture or video ingest + frame extractionclean: prune snapshot cache and temp filesconfig: init/show/edit/validate, providers, models, credentialsimage: capture screenshots (screen/window/menu bar regions)learn: print the full agent guide + tool cataloglist: apps, windows, screens, menubar, permissionspermissions: check Screen Recording/Accessibility statusrun: execute.peekaboo.jsonscriptssleep: pause execution for a durationtools: list available tools with filtering/display options
Interaction
click: target by ID/query/coords with smart waitsdrag: drag & drop across elements/coords/Dockhotkey: modifier combos likecmd,shift,tmove: cursor positioning with optional smoothingpaste: set clipboard -> paste -> restorepress: special-key sequences with repeatsscroll: directional scrolling (targeted + smooth)swipe: gesture-style drags between targetstype: text + control keys (--clear, delays)
System
app: launch/quit/relaunch/hide/unhide/switch/list appsclipboard: read/write clipboard (text/images/files)dialog: click/input/file/dismiss/list system dialogsdock: launch/right-click/hide/show/list Dock itemsmenu: click/list application menus + menu extrasmenubar: list/click status bar itemsopen: enhancedopenwith app targeting + JSON payloadsspace: list/switch/move-window (Spaces)visualizer: exercise Peekaboo visual feedback animationswindow: close/minimize/maximize/move/resize/focus/list
Vision
see: annotated UI maps, snapshot IDs, optional analysis
Global runtime flags
--json/-j,--verbose/-v,--log-level <level>--no-remote,--bridge-socket <path>
Quickstart (happy path)
peekaboo permissions
peekaboo list apps --json
peekaboo see --annotate --path /tmp/peekaboo-see.png
peekaboo click --on B1
peekaboo type "Hello" --return
Common targeting parameters (most interaction commands)
- App/window:
--app,--pid,--window-title,--window-id,--window-index - Snapshot targeting:
--snapshot(ID fromsee; defaults to latest) - Element/coords:
--on/--id(element ID),--coords x,y - Focus control:
--no-auto-focus,--space-switch,--bring-to-current-space,--focus-timeout-seconds,--focus-retry-count
Common capture parameters
- Output:
--path,--format png|jpg,--retina - Targeting:
--mode screen|window|frontmost,--screen-index,--window-title,--window-id - Analysis:
--analyze "prompt",--annotate - Capture engine:
--capture-engine auto|classic|cg|modern|sckit
Common motion/typing parameters
- Timing:
--duration(drag/swipe),--steps,--delay(type/scroll/press) - Human-ish movement:
--profile human|linear,--wpm(typing) - Scroll:
--direction up|down|left|right,--amount <ticks>,--smooth
Examples
See -> click -> type (most reliable flow)
peekaboo see --app Safari --window-title "Login" --annotate --path /tmp/see.png
peekaboo click --on B3 --app Safari
peekaboo type "user@example.com" --app Safari
peekaboo press tab --count 1 --app Safari
peekaboo type "supersecret" --app Safari --return
Target by window id
peekaboo list windows --app "Visual Studio Code" --json
peekaboo click --window-id 12345 --coords 120,160
peekaboo type "Hello from Peekaboo" --window-id 12345
Capture screenshots + analyze
peekaboo image --mode screen --screen-index 0 --retina --path /tmp/screen.png
peekaboo image --app Safari --window-title "Dashboard" --analyze "Summarize KPIs"
peekaboo see --mode screen --screen-index 0 --analyze "Summarize the dashboard"
Live capture (motion-aware)
peekaboo capture live --mode region --region 100,100,800,600 --duration 30 \
--active-fps 8 --idle-fps 2 --highlight-changes --path /tmp/capture
App + window management
peekaboo app launch "Safari" --open https://example.com
peekaboo window focus --app Safari --window-title "Example"
peekaboo window set-bounds --app Safari --x 50 --y 50 --width 1200 --height 800
peekaboo app quit --app Safari
Menus, menubar, dock
peekaboo menu click --app Safari --item "New Window"
peekaboo menu click --app TextEdit --path "Format > Font > Show Fonts"
peekaboo menu click-extra --title "WiFi"
peekaboo dock launch Safari
peekaboo menubar list --json
Mouse + gesture input
peekaboo move 500,300 --smooth
peekaboo drag --from B1 --to T2
peekaboo swipe --from-coords 100,500 --to-coords 100,200 --duration 800
peekaboo scroll --direction down --amount 6 --smooth
Keyboard input
peekaboo hotkey --keys "cmd,shift,t"
peekaboo press escape
peekaboo type "Line 1\nLine 2" --delay 10
Notes
- Requires Screen Recording + Accessibility permissions.
- Use
peekaboo see --annotateto identify targets before clicking.