verify-tests-fail-without-fix
Verifies UI tests catch the bug. Auto-detects mode based on git diff - if fix files exist, verifies FAIL without fix and PASS with fix. If only test files, verifies tests FAIL.
📖 中文导读:Verifies UI tests catch the bug. Auto-detects mode based on git diff - if fix files exist, verifies FAIL without fix and PASS with fix. If only test files, verifies tests FAIL. 以下正文为该技能的原始 SKILL.md 内容(英文),上方安装命令可直接使用。
name: verify-tests-fail-without-fix description: Verifies UI tests catch the bug. Supports two modes - verify failure only (test creation) or full verification (test + fix validation). metadata: author: dotnet-maui version: "1.0" compatibility: Requires git, PowerShell, and .NET SDK for building and running tests.
Verify Tests Fail Without Fix
Verifies UI tests actually catch the issue. Supports two workflow modes:
Mode 1: Verify Failure Only (Test Creation)
Use when creating tests before writing a fix:
- Runs tests to verify they FAIL (proving they catch the bug)
- No fix files required
- Perfect for test-first development
# Auto-detect test filter from changed test files
pwsh .github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 -Platform android
# With explicit test filter
pwsh .github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 -Platform ios -TestFilter "Issue33356"
Mode 2: Full Verification (Fix Validation)
Use when validating both tests and fix:
- Without fix - tests should FAIL (bug is present)
- With fix - tests should PASS (bug is fixed)
# Auto-detect everything (recommended)
pwsh .github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 -Platform android -RequireFullVerification
# With explicit test filter
pwsh .github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 -Platform ios -TestFilter "Issue33356" -RequireFullVerification
Note: -RequireFullVerification ensures the script errors if no fix files are detected, preventing silent fallback to failure-only mode.
Requirements
Verify Failure Only Mode:
- Test files in the PR (or working directory)
Full Verification Mode:
- Test files in the PR
- Fix files in the PR (non-test code changes)
The script auto-detects which mode to use based on whether fix files are present.
Expected Output
Verify Failure Only Mode:
╔═══════════════════════════════════════════════════════════╗
║ VERIFICATION PASSED ✅ ║
╠═══════════════════════════════════════════════════════════╣
║ Tests FAILED as expected! ║
║ This proves the tests correctly reproduce the bug. ║
╚═══════════════════════════════════════════════════════════╝
Full Verification Mode:
╔═══════════════════════════════════════════════════════════╗
║ VERIFICATION PASSED ✅ ║
╠═══════════════════════════════════════════════════════════╣
║ - FAIL without fix (as expected) ║
║ - PASS with fix (as expected) ║
╚═══════════════════════════════════════════════════════════╝
What It Does
Verify Failure Only Mode (no fix files):
- Fetches base branch from origin (if available)
- Auto-detects test classes from changed test files
- Runs tests (should FAIL to prove they catch the bug)
- Reports result
Full Verification Mode (fix files detected):
- Fetches base branch from origin to ensure accurate diff
- Auto-detects fix files (non-test code) from git diff
- Auto-detects test classes from
TestCases.Shared.Tests/*.cs - Reverts fix files to base branch
- Runs tests (should FAIL without fix)
- Restores fix files
- Runs tests (should PASS with fix)
- Reports result
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| No fix files detected | Base branch detection failed or no non-test files changed | Use -FixFiles or -BaseBranch explicitly |
| Tests pass without fix | Tests don't detect the bug | Review test assertions, update test |
| Tests fail with fix | Fix doesn't work or test is wrong | Review fix implementation |
| App crashes | Duplicate issue numbers, XAML error | Check device logs |
| Element not found | Wrong AutomationId, app crashed | Verify IDs match |
Optional Parameters
# Require full verification (fail if no fix files detected) - recommended
-RequireFullVerification
# Explicit test filter
-TestFilter "Issue32030|ButtonUITests"
# Explicit fix files
-FixFiles @("src/Core/src/File.cs")
# Explicit base branch
-BaseBranch "main"