qa-contract-validator

pcortes's avatarfrom pcortes

Validates API contracts by discovering consumers and ensuring responses match their expectations. Detects breaking changes.

⭐0stars🔀0forks📁View on GitHub🕐Updated Jan 2, 2026⬇️1downloads

When & Why to Use This Skill

The qa-contract-validator is an automated agent designed to ensure API stability and prevent integration failures. By scanning frontend code, service clients, and documentation, it discovers how consumers actually use your APIs and validates that responses meet those specific expectations. It proactively detects breaking changes such as renamed fields, type mismatches, or missing data, ensuring seamless communication between microservices and client applications.

Use Cases

  • Automated Breaking Change Detection: Scan pull requests to identify if API changes will break existing frontend components or downstream services before they reach production.
  • Consumer-Driven Contract Discovery: Automatically map out API dependencies by analyzing fetch/axios calls in React/Vue apps or HTTP client usage in Python/Java services.
  • Legacy System Migration: Validate that a refactored backend service still honors the implicit contracts expected by older client versions to ensure backward compatibility.
  • Documentation vs. Reality Audit: Compare actual code usage and integration tests against OpenAPI/Swagger specifications to find discrepancies in field nullability or data types.
  • Cross-Service Integration Testing: Verify that internal microservice communication remains intact by extracting and validating contracts from service-to-service HTTP requests.
nameqa-contract-validator
description>
allowed-toolsRead,Glob,Grep

Contract Validator Agent

You validate that API responses match what consumers expect.

Consumer Discovery

Find consumers by searching for:

1. Frontend Code

# Look for fetch/axios calls
Grep: "fetch\(.*api.*\)" in *.ts, *.tsx, *.js
Grep: "axios\.(get|post|put|delete)" in *.ts, *.tsx, *.js

2. Other Services

# Look for HTTP client usage
Grep: "requests\.(get|post)" in *.py
Grep: "HttpClient" in *.java
Grep: "http\.Get|http\.Post" in *.go

3. Integration Tests

# Tests often document expected contracts
Grep: "assert.*response" in test_*.py
Grep: "expect.*response" in *.test.ts

4. OpenAPI/Swagger Specs

Glob: **/openapi.yaml, **/openapi.json
Glob: **/swagger.yaml, **/swagger.json

Contract Extraction

From consumers, extract expectations:

Example: Frontend Consumer

// frontend/src/hooks/useUsers.ts
const response = await fetch('/api/users/123');
const user = await response.json();
console.log(user.name, user.email, user.createdAt);

Extracted contract:

{
  "endpoint": "GET /api/users/{id}",
  "consumer": "frontend/src/hooks/useUsers.ts:3",
  "expected_fields": ["name", "email", "createdAt"],
  "field_types": {
    "name": "string",
    "email": "string",
    "createdAt": "datetime"
  }
}

Example: Python Consumer

# service/client.py
resp = requests.get(f"{API_URL}/api/users/{user_id}")
data = resp.json()
return User(
    id=data["id"],
    name=data["name"],
    email=data["email"],
    active=data.get("active", True)
)

Extracted contract:

{
  "endpoint": "GET /api/users/{id}",
  "consumer": "service/client.py:15",
  "expected_fields": ["id", "name", "email"],
  "optional_fields": ["active"],
  "field_types": {
    "id": "integer",
    "name": "string",
    "email": "string",
    "active": "boolean"
  }
}

Validation Rules

  1. Field Presence: All expected fields must be present
  2. Field Types: Types must match (string, number, boolean, array, object)
  3. Nullability: If consumer doesn't handle null, field must not be null
  4. Field Names: Exact match (case-sensitive)
  5. Array Structure: Array items must match expected structure

Breaking Change Detection

Identify these breaking changes:

  • Removed fields: Field was in old response, missing in new
  • Type changes: Field type changed (e.g., string to number)
  • New required fields: New field that consumer doesn't expect
  • Renamed fields: Field name changed (detected by type similarity)
  • Nullability changes: Previously non-null field now nullable

Graceful Degradation

If no consumers found:

  1. Log warning: "No consumers discovered for {endpoint}"
  2. Skip contract validation for that endpoint
  3. Continue with other endpoints
  4. Report in output: "contracts_skipped": ["GET /api/users"]

Output Format

{
  "agent": "contract_validator",
  "contracts_checked": 8,
  "contracts_valid": 7,
  "contracts_broken": 1,
  "contracts_skipped": 0,
  "findings": [
    {
      "id": "CV-001",
      "severity": "critical",
      "endpoint": "GET /api/users/{id}",
      "consumer": "frontend/src/components/UserProfile.tsx:45",
      "issue_type": "field_renamed",
      "title": "Breaking change: field renamed",
      "description": "Field 'createdAt' renamed to 'created_at'",
      "evidence": {
        "consumer_expects": "createdAt",
        "api_returns": "created_at",
        "consumer_code": "const created = user.createdAt;"
      },
      "recommendation": "Keep 'createdAt' for backwards compatibility, or update all consumers"
    }
  ],
  "confidence": 0.90
}

Severity Guidelines

  • Critical: Removed required field, type change, renamed field
  • Moderate: New required field, nullability change
  • Minor: Added optional field (usually not breaking)

Multiple Consumers

When multiple consumers expect different things:

  1. Report all consumers with their expectations
  2. Flag inconsistencies between consumers
  3. Validate API against union of all expectations
  4. Let human decide which expectation is correct