quality-checker

jmagly's avatarfrom jmagly

Validate skill quality, completeness, and adherence to standards. Use before packaging to ensure skill meets quality requirements. Use when relevant to the task.

⭐51stars🔀4forks📁View on GitHub🕐Updated Jan 4, 2026⬇️2downloads

When & Why to Use This Skill

The Quality Checker skill is a professional-grade validation tool designed to automate the quality assurance process for Claude skill packages. It rigorously evaluates skill directories for structural integrity, documentation completeness, and code example validity, ensuring that all agentic components meet production standards before deployment.

Use Cases

  • Pre-deployment Validation: Automatically verify that a Claude skill package contains all required files, such as SKILL.md and reference directories, to prevent packaging errors.
  • Documentation Auditing: Scan SKILL.md files to ensure they include essential sections like Purpose, Grounding Checkpoints, and Navigation, while maintaining high content density.
  • Code Quality Enforcement: Validate that documentation contains a sufficient number of language-tagged, functional code examples rather than just placeholders.
  • Automated Quality Reporting: Generate comprehensive markdown reports with weighted scores across dimensions like Structure, Content, and Documentation to provide actionable feedback for developers.
  • CI/CD Pipeline Integration: Use the skill as a gatekeeper in agent development workflows to ensure only high-scoring, standardized skills move from enhancement to packaging.
namequality-checker
descriptionValidate skill quality, completeness, and adherence to standards. Use before packaging to ensure skill meets quality requirements. Use when relevant to the task.

Quality Checker Skill

Purpose

Single responsibility: Validate Claude skill packages for quality, completeness, and standards compliance before upload. (BP-4)

Grounding Checkpoint (Archetype 1 Mitigation)

Before executing, VERIFY:

  • Skill directory exists
  • SKILL.md is present
  • Quality criteria are defined
  • Validation scope is clear (quick/full/custom)

DO NOT validate without defining quality criteria.

Uncertainty Escalation (Archetype 2 Mitigation)

ASK USER instead of guessing when:

  • Quality threshold unclear (strict vs lenient)
  • Custom validation rules needed
  • Failures found - block or warn?
  • Edge cases in validation logic

NEVER auto-pass quality checks without proper validation.

Context Scope (Archetype 3 Mitigation)

Context Type Included Excluded
RELEVANT Skill directory, quality criteria Other skills
PERIPHERAL Quality examples for comparison Source documentation
DISTRACTOR Build process Enhancement history

Quality Dimensions

Dimension Weight Checks
Structure 25% Required files, directory layout
Content 35% SKILL.md completeness, references
Code Examples 20% Presence, syntax, relevance
Documentation 20% Clarity, navigation, completeness

Workflow Steps

Step 1: Structure Validation (Grounding)

# Required files
SKILL_DIR="output/<skill-name>"

# Check SKILL.md
test -f "$SKILL_DIR/SKILL.md" && echo "✅ SKILL.md present" || echo "❌ SKILL.md missing"

# Check references directory
test -d "$SKILL_DIR/references" && echo "✅ references/ present" || echo "❌ references/ missing"

# Check at least one reference file
ls "$SKILL_DIR/references/"*.md >/dev/null 2>&1 && \
  echo "✅ Reference files present" || echo "❌ No reference files"

# Check for index
test -f "$SKILL_DIR/references/index.md" && \
  echo "✅ Index present" || echo "⚠️ No index.md (recommended)"

Step 2: SKILL.md Content Validation

SKILL_MD="output/<skill-name>/SKILL.md"

# Required sections
echo "=== Section Check ==="
grep -q "^# " "$SKILL_MD" && echo "✅ Title present" || echo "❌ Missing title"
grep -q "^## Description\|^## Purpose" "$SKILL_MD" && echo "✅ Description present" || echo "❌ Missing description"

# Recommended sections
grep -q "^## Quick Reference\|^## Overview" "$SKILL_MD" && echo "✅ Quick reference" || echo "⚠️ No quick reference"
grep -q "^## Code Examples\|^## Examples" "$SKILL_MD" && echo "✅ Examples section" || echo "⚠️ No examples section"
grep -q "^## Navigation\|^## Contents" "$SKILL_MD" && echo "✅ Navigation" || echo "⚠️ No navigation"

# Content quality
echo ""
echo "=== Content Metrics ==="
echo "Lines: $(wc -l < "$SKILL_MD")"
echo "Code blocks: $(grep -c '```' "$SKILL_MD")"
echo "Sections: $(grep -c '^## ' "$SKILL_MD")"
echo "Links: $(grep -oE '\[.*\]\(.*\)' "$SKILL_MD" | wc -l)"

Step 3: Code Example Validation

SKILL_MD="output/<skill-name>/SKILL.md"

# Extract code blocks
echo "=== Code Examples ==="
example_count=$(grep -c '```' "$SKILL_MD")
echo "Total code blocks: $((example_count / 2))"

# Check for language tags
tagged=$(grep -c '```[a-z]' "$SKILL_MD")
echo "Language-tagged blocks: $tagged"

# Check code isn't just placeholders
placeholder_count=$(grep -E '```\n(# placeholder|// TODO|pass)\n```' "$SKILL_MD" | wc -l)
echo "Placeholder blocks: $placeholder_count"

# Minimum requirement: 3 real code examples
real_examples=$((example_count / 2 - placeholder_count))
if [ "$real_examples" -ge 3 ]; then
  echo "✅ Sufficient code examples ($real_examples)"
else
  echo "⚠️ Few code examples ($real_examples, recommend 3+)"
fi

Step 4: Reference Quality Validation

REF_DIR="output/<skill-name>/references"

echo "=== Reference Files ==="
for file in "$REF_DIR"/*.md; do
  if [ -f "$file" ]; then
    lines=$(wc -l < "$file")
    name=$(basename "$file")
    if [ "$lines" -lt 10 ]; then
      echo "⚠️ $name: $lines lines (sparse)"
    else
      echo "✅ $name: $lines lines"
    fi
  fi
done

# Total reference content
total_lines=$(cat "$REF_DIR"/*.md 2>/dev/null | wc -l)
echo ""
echo "Total reference content: $total_lines lines"

Step 5: Generate Quality Report

# Quality Report: <skill-name>

## Summary
- Overall Score: XX/100
- Status: PASS/WARN/FAIL

## Structure (25/25)
- [x] SKILL.md present
- [x] references/ directory
- [x] Reference files present
- [ ] Optional: scripts/, assets/

## Content (30/35)
- [x] Title present
- [x] Description clear
- [x] Quick reference
- [ ] FAQ section (missing)

## Code Examples (15/20)
- [x] 5 code examples
- [x] Language tags
- [ ] Example diversity (all Python)

## Documentation (18/20)
- [x] Navigation table
- [x] Links work
- [ ] Version info missing

## Recommendations
1. Add FAQ section based on common questions
2. Include examples in other languages
3. Add version/last updated info

Recovery Protocol (Archetype 4 Mitigation)

On error:

  1. PAUSE - Complete partial validation
  2. DIAGNOSE - Check error type:
    • File not found → Check path
    • Parse error → Check file format
    • Script error → Simplify validation
  3. ADAPT - Adjust validation scope
  4. RETRY - With corrected parameters (max 3 attempts)
  5. ESCALATE - Report partial results

Checkpoint Support

State saved to: .aiwg/working/checkpoints/quality-checker/

checkpoints/quality-checker/
├── structure_results.json
├── content_results.json
├── code_results.json
├── docs_results.json
└── final_report.md

Quality Thresholds

Level Score Action
PASS 80-100 Ready for packaging
WARN 60-79 Review recommendations
FAIL <60 Address issues before packaging

Configuration Options

{
  "skill_dir": "output/myskill/",
  "validation_level": "full",
  "thresholds": {
    "pass": 80,
    "warn": 60
  },
  "requirements": {
    "min_skill_md_lines": 100,
    "min_code_examples": 3,
    "min_reference_files": 2,
    "require_navigation": true,
    "require_faq": false
  },
  "output": {
    "report_format": "markdown",
    "save_report": true
  }
}

Validation Levels

Level Checks Time
quick Structure only <5s
standard Structure + content <30s
full All dimensions <2m
strict Full + extra rules <5m

Custom Validation Rules

Add custom rules via configuration:

{
  "custom_rules": [
    {
      "name": "api_coverage",
      "type": "grep",
      "pattern": "^### .*\\(\\)",
      "file": "references/api.md",
      "min_matches": 10,
      "message": "API reference should document at least 10 functions"
    }
  ]
}

Troubleshooting

Issue Diagnosis Solution
False positives Rules too strict Adjust thresholds
Missed issues Rules too lenient Use strict mode
Slow validation Full mode on large skill Use quick mode first
Parse errors Malformed markdown Fix source files

Integration with Workflow

doc-scraper → skill-builder → skill-enhancer → quality-checker → skill-packager
                                                     ↓
                                              [If FAIL: fix issues]
                                                     ↓
                                              [If WARN: review]
                                                     ↓
                                              [If PASS: package]

References

  • Claude Skills Quality Guidelines: https://docs.anthropic.com/skills
  • AIWG Quality Standards: agentic/code/addons/writing-quality/
  • REF-001: Production-Grade Agentic Workflows (BP-4, BP-9)
  • REF-002: LLM Failure Modes (Archetype 1 grounding, Archetype 2 over-helpfulness)