agentic-kpi-tracking

Track and measure agentic coding KPIs for ZTE progression. Use when measuring workflow effectiveness, tracking Size/Attempts/Streak/Presence metrics, or assessing readiness for autonomous operation.

⭐3stars🔀0forks📁View on GitHub🕐Updated Jan 3, 2026

When & Why to Use This Skill

This Claude skill provides a robust framework for tracking and measuring agentic coding Key Performance Indicators (KPIs) to facilitate Zero-Touch Engineering (ZTE) progression. It enables developers and teams to quantitatively assess AI agent performance, workflow effectiveness, and readiness for autonomous operation by monitoring critical metrics such as success streaks, attempt counts, and code change magnitude.

Use Cases

  • ZTE Readiness Assessment: Determine if an AI agent is stable enough for unsupervised operation by analyzing success streaks and average presence metrics.
  • Workflow Efficiency Analysis: Identify technical bottlenecks in agentic processes by tracking the number of attempts required to reach a successful code patch.
  • Performance Trend Monitoring: Visualize the evolution of agent capabilities over time, measuring its ability to handle increasing task complexity and larger plan sizes.
  • Engineering Impact Reporting: Quantify the tangible contributions of AI agents to a codebase by monitoring total diff sizes, lines added/removed, and files modified per run.
nameagentic-kpi-tracking
descriptionTrack and measure agentic coding KPIs for ZTE progression. Use when measuring workflow effectiveness, tracking Size/Attempts/Streak/Presence metrics, or assessing readiness for autonomous operation.
allowed-toolsRead, Grep, Glob

Agentic KPI Tracking Skill

Guide measurement and tracking of agentic coding KPIs to assess ZTE readiness.

When to Use

  • Measuring agentic workflow effectiveness
  • Tracking progress toward ZTE
  • Analyzing success patterns
  • Identifying improvement areas

Core KPIs

Summary Metrics

Metric Calculation Target
Current Streak Consecutive successes (Attempts <= 2) Higher is better
Longest Streak Best consecutive success run Track improvement
Average Presence Mean attempts across all runs Target: 1
Total Plan Size Sum of all plan sizes Track scaling
Total Diff Size Sum of all changes (added + removed) Track throughput

Per-Run Metrics

Metric Source Meaning
Attempts Count of plan/patch runs 1 = perfect, higher = retries
Plan Size Lines in plan file Task complexity
Diff Size Lines added + removed Change magnitude
Files Changed Number of files modified Change scope

Calculation Methods

Attempts Count

Only count workflow restarts:

attempts_incrementing = ["adw_plan_iso", "adw_patch_iso"]
attempts = count(workflow in all_adws if workflow in attempts_incrementing)

Build/test/review don't increment - only full replans.

Streak Calculation

current_streak = 0
for run in reversed(runs):
    if run.attempts <= 2:
        current_streak += 1
    else:
        break

Diff Statistics

git diff origin/main --shortstat
# Output: X files changed, Y insertions(+), Z deletions(-)

KPI File Format

Store in app_docs/agentic_kpis.md or equivalent:

# Agentic KPIs

## Summary

| Metric | Value |
| --- | --- |
| Current Streak | 5 |
| Longest Streak | 12 |
| Average Presence | 1.3 |
| Total Plan Size | 450 lines |
| Total Diff Size | 2,340 lines |

## Detail

| Date | ADW ID | Issue | Class | Attempts | Plan Size | Diff +/- | Files |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 2024-01-15 | abc123 | #45 | /bug | 1 | 35 | +45/-12 | 3 |
| 2024-01-14 | def456 | #44 | /feature | 2 | 85 | +120/-30 | 8 |

Tracking Workflow

Step 1: Gather Current Run Data

From state or git:

  • ADW ID
  • Issue number
  • Issue classification
  • Plan file path
  • All workflows run (for attempts)

Step 2: Calculate Metrics

attempts = count_attempts(all_adws)
plan_size = wc_lines(plan_file)
diff_stats = parse_git_diff()

Step 3: Update Detail Table

Add new row with current run data.

Step 4: Recalculate Summary

Update all summary metrics based on full detail table.

  • Is streak increasing?
  • Is average presence decreasing?
  • Are plan sizes growing (handling bigger tasks)?

ZTE Readiness Indicators

Based on KPIs, assess ZTE readiness:

Indicator Threshold Status
Current Streak >= 5 Ready to try ZTE
Average Presence <= 1.5 Good efficiency
Recent Failures 0 in last 10 High confidence
Plan Size Trend Increasing Scaling up

Key Memory References

  • @agentic-kpis.md - KPI definitions from Lesson 002
  • @zte-progression.md - How KPIs relate to ZTE levels
  • @zte-confidence-building.md - Using KPIs for confidence

Output Format

Provide KPI update:

## KPI Update

**Run:** {adw_id}
**Issue:** #{issue_number} ({issue_class})

### This Run
- Attempts: 1
- Plan Size: 45 lines
- Diff: +67/-23 (4 files)

### Updated Summary
- Current Streak: 6 (was 5)
- Longest Streak: 12 (unchanged)
- Average Presence: 1.28 (improved)

### Analysis
[Trend observations and recommendations]

Anti-Patterns

  • Gaming metrics (easy tasks only)
  • Ignoring failures (not counting retries)
  • Not tracking consistently
  • Celebrating streaks over actual delivery

Version History

  • v1.0.0 (2025-12-26): Initial release

Last Updated

Date: 2025-12-26 Model: claude-opus-4-5-20251101