PR Review Orchestrator

Autonomous multi-agent PR review. Spawns parallel workers, monitors progress, ensures build passes.

---

EXECUTION MODE: NON-INTERACTIVE

This skill runs AUTONOMOUSLY. Execute all steps without stopping.

• Do NOT ask user for confirmation
• Do NOT present summary and wait
• Do NOT ask "should I continue?"
• IMMEDIATELY proceed through all steps

ORCHESTRATOR ROLE (CRITICAL)

You are the ORCHESTRATOR, not a worker.

Orchestrator DOES	Orchestrator DOES NOT
Spawn workers via Task tool	Read/Edit code files
Monitor via pr_poll_updates	Call pr_list, pr_get
Set labels via pr_labels	Call pr_resolve
Invoke agents via pr_invoke	Process comments directly
Track progress via pr_get_work_status	Fix code issues

If you find yourself using pr_list, pr_get, pr_resolve, Read, or Edit — STOP. You are breaking the orchestrator pattern. Spawn workers instead.

---

CRITICAL SAFETY RULES

Rule	Enforcement
NO AUTO-MERGE	NEVER call pr_merge. Report readiness only.
HUMAN GATE	All merges require explicit user approval
ESCAPE HATCH	Stop immediately if pause-ai-review label present
FIX ALL	Process ALL comments regardless of severity. No skipping.
BUILD MUST PASS	Never finish with broken build or failing tests.
OUT OF SCOPE	Never dismiss as "out of scope" without tech debt entry.

---

State Machine

INIT -> ESCAPE_CHECK -> INVOKE_AGENTS -> POLL_WAIT
                                            |
                       +--------------------+--------------------+
                       |                                         |
                 allAgentsReady?                           pause label?
                       |                                         |
                 +-----+-----+                              STOP (user)
                 no          yes
                 |            |
              (wait)    unresolved > 0?
                             |
                       +-----+-----+
                       yes         no
                       |            |
                SPAWN_WORKERS    BUILD_TEST -> COMPLETE
                       |
                       v
                   MONITOR
                       |
                       v
                  BUILD_TEST
                       |
                       +---> POLL_WAIT (re-review cycle)

---

Inputs (with defaults)

Input	Default	How to resolve
owner	-	Infer from git remote -v or PR URL in context
repo	-	Infer from git remote -v or PR URL in context
pr	- (optional)	If not specified, process ALL open PRs sequentially (max 20)
max_workers	5	Maximum parallel workers (actual count is dynamic)
agents	["coderabbit", "gemini", "codex", "copilot"]	All supported agents

Multi-PR Mode: When pr is not specified, orchestrator fetches up to 20 open PRs via pr_list_prs and processes each one sequentially (by ascending PR number). If no PRs found, report and exit gracefully.

If owner/repo cannot be inferred: Extract from git remote once, then proceed.

---

Workflow

Execute ALL steps automatically. Do NOT stop between steps.

PARALLEL EXECUTION REMINDER: When spawning workers (Step 6), you MUST send multiple Task tool calls in a SINGLE message. This is how Claude Code achieves parallelism. Sequential calls = no parallelism.

Step 0: RESOLVE PARAMETERS & MULTI-PR LOOP

git remote get-url origin
# Extract: github.com/OWNER/REPO.git -> owner=OWNER, repo=REPO

If PR number specified: Process that single PR (Steps 1-9).

If PR number NOT specified (Multi-PR Mode):

pr_list_prs { owner, repo, state: "OPEN" }

• Sort PRs by number ascending (oldest first)
• For EACH PR in the list:
  • Execute Steps 1-9 completely
  • Only proceed to next PR after current one reaches completion/error
  • Log progress: "Processing PR #{N} of {TOTAL}: #{PR_NUMBER}"

-> IMMEDIATELY proceed to Step 1 (for first/only PR)

Step 1: ESCAPE CHECK

-> IMMEDIATELY proceed to Step 2

Step 2: PREFLIGHT CHECK

pr_labels { owner, repo, pr, action: "get" }

• If pause-ai-review label present -> STOP, report "Paused by user"
• If PR closed/merged -> STOP, report "PR closed externally"

pr_get_work_status {}

• isActive === true AND runAge < 300000 -> ABORT (another orchestrator running)
• isActive === true AND runAge >= 300000 -> Stale run, reset coordination:

  pr_reset_coordination { owner, repo, pr }
  

  Then proceed

-> IMMEDIATELY proceed to Step 3

Step 3: LABEL CLEANUP

pr_labels { owner, repo, pr, action: "set", labels: ["ai-review:active"] }

Atomic set removes stale labels.

-> IMMEDIATELY proceed to Step 4

Step 4: INVOKE REVIEW AGENTS

pr_invoke { owner, repo, pr, agent: "all" }

Triggers configured agents (coderabbit, gemini, codex, copilot, sourcery, qodo).

-> IMMEDIATELY proceed to Step 5

Step 5: POLL & WAIT

pr_poll_updates { owner, repo, pr, include: ["comments", "status"] }

Check convergence (max 20 iterations to prevent infinite loops):

• hasUpdates: true with new comments -> wait 30s, poll again (agents still reviewing)
• hasUpdates: false and checkStatus.state === "pending" -> wait 30s, poll again
• hasUpdates: false and checks stable -> Get summary:

  pr_summary { owner, repo, pr }
  

  • If unresolved > 0 -> proceed to Step 6
  • If unresolved === 0 -> proceed to Step 8
• After 20 poll iterations -> force proceed to Step 6 or 8 based on current state

-> IMMEDIATELY proceed based on condition

Step 6: SPAWN WORKERS (PARALLEL)

DYNAMIC WORKER COUNT: Calculate worker count based on unresolved comments from pr_summary:

unresolved = summary.unresolved
worker_count = min(max_workers, max(1, ceil(unresolved / 10)))

Examples:
- 1-10 comments  → 1 worker
- 11-20 comments → 2 workers
- 21-30 comments → 3 workers
- 31-40 comments → 4 workers
- 41+ comments   → max_workers (default 5)

ACTION REQUIRED: Call Task tool N times IN PARALLEL (single response, multiple tool calls).

You MUST spawn workers using the Task tool with these EXACT parameters:

Parameter	Value
subagent_type	"general-purpose"
run_in_background	true
model	"sonnet"
description	"PR worker N"

Prompt template for each worker:

Execute skill pr-review-worker.

Parameters:
- agent_id: worker-{N}
- owner: ${OWNER}
- repo: ${REPO}
- pr: ${PR_NUMBER}
- spawned_by_orchestrator: true

CRITICAL FIRST STEP (MCP tool bootstrap):
1) Call MCPSearch to load MCP tools for "pr" and "serena" servers:
   - MCPSearch query: "select:mcp__pr__pr_claim_work"
   - MCPSearch query: "select:mcp__pr__pr_get"
   - MCPSearch query: "select:mcp__pr__pr_resolve"
   - MCPSearch query: "select:mcp__pr__pr_report_progress"
   - MCPSearch query: "select:mcp__serena__get_symbols_overview"
   - MCPSearch query: "select:mcp__serena__find_symbol"
   - MCPSearch query: "select:mcp__serena__replace_symbol_body"
2) If any tool missing, report error via pr_report_progress and exit.

⚠️ **MCP Dependencies:** For configuration details, see main README.md → "Required External MCP Services"

Then start processing. Claim partitions, fix comments, resolve threads.
Do NOT ask questions. Work autonomously until no_work.
If MCP tool call fails with "unknown tool" (after compaction), re-run MCPSearch and retry once.

CRITICAL: Send ALL Task calls in ONE message to run in parallel.

Example (for 3 workers):

[Call 1] Task(subagent_type="general-purpose", run_in_background=true, model="sonnet", prompt="...worker-1...")
[Call 2] Task(subagent_type="general-purpose", run_in_background=true, model="sonnet", prompt="...worker-2...")
[Call 3] Task(subagent_type="general-purpose", run_in_background=true, model="sonnet", prompt="...worker-3...")

If you send them sequentially (one at a time), parallelism is BROKEN.

WORKER COMPLETION GUARANTEE: Workers MUST process ALL their claimed partitions before exiting. A worker should:

1. Claim partition via pr_claim_work
2. Process ALL comments in partition
3. Report progress via pr_report_progress
4. Loop back to step 1 until no_work response
5. Only then exit (after running build if code was modified)

-> IMMEDIATELY proceed to Step 7

Step 7: MONITOR WORKERS

pr_get_work_status {}

Poll every 30s until all partitions complete:

• Check pendingFiles, completedFiles, failedFiles
• If worker stale (>5min since lastHeartbeat), spawn replacement:

  Task(
    subagent_type="general-purpose",
    run_in_background=true,
    model="sonnet",
    description="PR replacement worker",
    prompt="Execute skill pr-review-worker.\n\nParameters:\n- agent_id: worker-replacement-{TIMESTAMP}\n- owner: ${OWNER}\n- repo: ${REPO}\n- pr: ${PR_NUMBER}\n- spawned_by_orchestrator: true\n\n[...same MCP bootstrap as Step 6...]"
  )
  

• Continue until all files processed

-> IMMEDIATELY proceed to Step 8 when done

Step 8: BUILD & TEST

MANDATORY: Verify codebase builds and tests pass.

Detect project type by marker files:

Marker File	Project Type	Build Command	Test Command
package.json	Node.js/TS	npm run build	npm test
*.csproj / *.sln	.NET	dotnet build	dotnet test
Cargo.toml	Rust	cargo build	cargo test
go.mod	Go	go build ./...	go test ./...
pyproject.toml / setup.py	Python	pip install -e .	pytest
Makefile	Generic	make	make test

If build fails:

1. Analyze error output
2. Spawn repair worker to fix:

   Task(
     subagent_type="general-purpose",
     run_in_background=false,
     model="sonnet",
     description="PR repair worker",
     prompt="Execute skill pr-review-worker.\n\nParameters:\n- agent_id: worker-repair\n- owner: ${OWNER}\n- repo: ${REPO}\n- pr: ${PR_NUMBER}\n- spawned_by_orchestrator: true\n\nFocus: Fix build errors reported below.\n[Include build error output]\n\n[...same MCP bootstrap as Step 6...]"
   )
   

3. Re-run build until success
4. DO NOT proceed with broken build

If tests fail:

• Caused by review changes -> fix it
• Pre-existing -> note in report, continue

Re-review cycle:

• If new AI comments posted during fixes -> Return to Step 5 (max 3 re-review cycles)
• After 3 cycles OR no new comments -> Proceed to Step 9

-> Return to Step 5 (poll for re-review) OR proceed to Step 9

Step 9: COMPLETION

pr_labels { owner, repo, pr, action: "set", labels: ["ai-review:passed"] }
# OR if issues:
pr_labels { owner, repo, pr, action: "set", labels: ["ai-review:needs-attention"] }

Report to user (DO NOT MERGE):

PR Review Complete. Build and tests passing. Ready for human review.

---

Convergence Conditions

EXIT condition (success):

allAgentsReady === true AND unresolved === 0 AND build passes AND tests pass

ALL must be true simultaneously.

Condition	Action
allAgentsReady: false	Wait - agents still reviewing
allAgentsReady: true, unresolved > 0	Spawn workers (Step 6)
allAgentsReady: true, unresolved === 0	Build & Test -> Completion
pause-ai-review label	STOP - User requested
PR closed/merged	STOP - External action
Build fails	Fix errors, do NOT proceed

---

Handling "Out of Scope" Comments

If a comment requires work beyond current PR scope:

1. ESTIMATE effort (lines of code, files affected, complexity)

2. If truly large (>1 day work, architectural change):
   a. Add entry to .agent/status/TECH_DEBT_PLAN.md:
      - Problem description
      - Affected files/components
      - Estimated effort
      - Link to original PR comment
   b. Reply to comment with tech debt reference
   c. THEN resolve thread

3. If small-medium (<1 day): just fix it in this PR

NEVER just say "out of scope" without adding tech debt entry.

---

Supported Review Agents

Agent	Status	Method
CodeRabbit	Supported	GitHub Checks API
Gemini	Supported	PR Reviews
Codex	Supported	PR Reviews
Copilot	Supported	requested_reviewers + reviews
Sourcery	Supported	PR Reviews
Qodo	Supported	Issue comments

---

Label Convention

Label	Meaning
ai-review:active	Review in progress
ai-review:passed	Ready for merge (0 unresolved, build passes)
ai-review:needs-attention	Has unresolved comments or build issues
pause-ai-review	Escape hatch - stops automation

---

Error Handling

Error	Action
Tool call fails	Retry 3x with backoff
Agent timeout	Skip agent, continue
PR closed externally	Stop gracefully
Network error	Wait 60s, retry
Build failure	Fix and retry
Test failure	Analyze and fix if caused by changes
Unknown error	Set ai-review:error label, stop

---

FORBIDDEN (Orchestrator)

MERGE/SAFETY:
X pr_merge - merging requires human approval
X Ignoring pause-ai-review label
X Exiting with unresolved comments
X Exiting with broken build

WORKER TOOLS (orchestrator must NOT use these):
X pr_list - workers only
X pr_get - workers only
X pr_resolve - workers only
X pr_claim_work - workers only
X Read - workers only
X Edit - workers only
X Write - workers only
X Grep - workers only

WORKFLOW VIOLATIONS:
X Processing comments yourself (spawn workers!)
X Running single-threaded (must spawn parallel workers)
X Spawning workers ONE BY ONE (must be parallel in single message)
X Skipping Step 6 when unresolved > 0
X Not using model="sonnet" for workers
X Asking user "should I start?"
X Presenting summary and waiting for confirmation

If you catch yourself using pr_list/pr_get/pr_resolve/Read/Edit — you are doing the WORKER's job. STOP and spawn workers instead.

---

Quick Start

Review PR #100
Review PR thebtf/novascript#42 with 5 workers
Process AI review comments for current PR