agent-evaluator

arisng's avatarfrom arisng

Deterministic custom subagent selection helper. Use when you need a reproducible, auditable decision on which custom subagents to activate for a user query (runs scripts/agent_evaluator.py).

⭐1stars🔀0forks📁View on GitHub🕐Updated Dec 20, 2025
[Tool Routing and Orchestration]

When & Why to Use This Skill

The Agent Evaluator is a deterministic tool designed to streamline multi-agent workflows by intelligently selecting the most relevant subagents for any given user query. It provides auditable, score-based decisions to ensure the right specialized agents are activated, enhancing the precision and reliability of complex AI agent systems.

Use Cases

  • Multi-agent system orchestration: Automatically routing user requests to specialized subagents (e.g., coding, research, or data analysis agents) based on query intent.
  • Workflow auditing and debugging: Using the deterministic JSON output to understand exactly why specific agents were triggered or ignored during a complex task.
  • Context-aware agent activation: Evaluating queries alongside specific file contexts to ensure the subagent with the most relevant domain expertise is prioritized for the task.
nameagent-evaluator
descriptionDeterministic custom subagent selection helper. Use when you need a reproducible, auditable decision on which custom subagents to activate for a user query (runs scripts/agent_evaluator.py).

Agent Evaluator

Evaluate a user query against the workspace's available subagents and return a JSON decision payload (activated/required/suggested agents and scoring).

Mechanism

Run the evaluator script (located in scripts folder relative this skill file) with the user query as an argument.:

python scripts/agent_evaluator.py "YOUR_QUERY_HERE"

Optional: include a contextual file path as the second argument:

python scripts/agent_evaluator.py "YOUR_QUERY_HERE" "path/to/file.ext"

Output

  • Writes a JSON object to stdout.
  • Key fields include:
    • activated_agents
    • required_agents
    • suggested_agents
    • evaluations (per-agent score + reasoning)

Examples

Evaluate a query:

python scripts/agent_evaluator.py "Please help refine our custom instruction file"

Evaluate a query with file context:

python scripts/agent_evaluator.py "Update this instruction" "instructions/agent-forced-eval.instructions.md"