extract-test-failures

Extract and summarize test failures from logs. Use to quickly understand what tests failed and why.

⭐6stars🔀1forks📁View on GitHub🕐Updated Dec 11, 2025

When & Why to Use This Skill

This Claude skill automates the extraction and summarization of test failures from complex log files. It streamlines the debugging process by identifying specific error types—such as assertion failures, runtime errors, and timeouts—and providing a concise report. By transforming raw log data into actionable insights, it helps developers and QA engineers significantly reduce the time spent on manual log analysis and root cause identification in CI/CD pipelines.

Use Cases

  • CI/CD Failure Analysis: Automatically parse logs from failed Jenkins or GitHub Actions builds to provide a quick summary of which tests failed and why.
  • Regression Test Reporting: Summarize results from large-scale regression suites to identify patterns in failures and prioritize fixes.
  • Automated PR Feedback: Generate detailed pull request comments that highlight specific test failures and stack traces for faster code reviews.
  • Error Categorization: Group failures by type (e.g., logic errors vs. environment timeouts) to help assign debugging tasks to the appropriate team members.
  • Historical Failure Tracking: Extract failure data over time to identify flaky tests or recurring infrastructure issues.
nameextract-test-failures
description"Extract and summarize test failures from logs. Use to quickly understand what tests failed and why."
categorytesting
mcp_fallbacknone
user-invocablefalse

Extract and Summarize Test Failures

Parse test logs to extract failure information and create summary.

When to Use

  • CI test run failed, need to understand failures
  • Large test suite with many failures, need summary
  • Categorizing failures by type (assertion, runtime, timeout)
  • Creating PR review comments about failures
  • Tracking which tests are failing over time

Quick Reference

# Extract failed test names
grep "FAILED" test_output.log

# Get failure details with context
grep -A 10 "FAILED\|Error\|AssertionError" test_output.log

# Count failures by type
grep "FAILED\|AssertionError\|ValueError\|TypeError" test_output.log | sort | uniq -c

# Extract test summary line
tail -20 test_output.log | grep -E "passed|failed|error"

# Get specific failure info
grep -B 5 "AssertionError" test_output.log | head -50

Failure Extraction Workflow

  1. Collect log: Get test output from CI or local run
  2. Find failures: Extract FAILED markers and error types
  3. Group by type: Categorize assertion vs runtime vs timeout
  4. Extract details: Get error messages and stack traces
  5. Count totals: Summary of how many failures
  6. Identify patterns: Common failure causes
  7. Summarize: Create concise failure report

Failure Categories

Assertion Failures:

  • AssertionError - Expected value check failed
  • assert_equal() - Values don't match
  • assert_true() - Condition not true
  • Fix: Check test logic and expected values

Type/Attribute Errors:

  • AttributeError - Missing method/attribute
  • TypeError - Wrong argument type
  • ValueError - Invalid value for operation
  • Fix: Check code syntax and types

Runtime Errors:

  • IndexError - Out of bounds access
  • KeyError - Dictionary key not found
  • ZeroDivisionError - Division by zero
  • Fix: Check boundary conditions

Timeout/Hanging:

  • Test takes too long
  • Infinite loop or deadlock
  • Resource exhaustion
  • Fix: Optimize or add timeout

Output Format

Report failures with:

  1. Total Failures - Count and percentage of total tests
  2. Failure Summary - By type (assertion, runtime, timeout, other)
  3. Failed Tests - List of test names that failed
  4. Top Issues - Most common failure patterns
  5. Error Messages - Representative error snippets
  6. Recommendations - Which tests to focus on first

Error Handling

Problem Solution
No FAILED markers Check log format, may use different pattern
Truncated output Get full log from artifacts instead of view
Mixed output types Filter by log level or timestamp
Encoding issues Convert to UTF-8 first
Very large logs Split and process in chunks

References

  • See test-diff-analyzer for identifying flaky tests
  • See analyze-ci-failure-logs for CI-specific failures
  • See generate-fix-suggestions for automated fixes