smoke-suite-debugging
Use this when smoke tests fail (binary/web assets, Playwright setup, server lifecycle, ports, env vars).
When & Why to Use This Skill
This Claude skill is designed to streamline the debugging process for automated smoke test suites. It provides expert guidance on resolving common failures related to Playwright configurations, binary asset paths, server lifecycle management, and environment variable conflicts, ensuring a stable and reliable CI/CD pipeline.
Use Cases
- Diagnosing 'Binary not found' errors by verifying build artifacts and configuring correct binary paths (e.g., LOTAR_BINARY_PATH).
- Resolving port conflicts and 'Port already in use' errors by managing server lifecycles and identifying stuck processes during test execution.
- Troubleshooting flaky smoke tests caused by SSE (Server-Sent Events) readiness issues or timing mismatches in server heartbeats.
- Automating the setup and verification of Playwright browser environments to ensure test readiness across different development machines.
- Optimizing local debugging loops using Vitest and in-band execution to isolate specific test suite failures.
| name | smoke-suite-debugging |
|---|---|
| description | Use this when smoke tests fail (binary/web assets, Playwright setup, server lifecycle, ports, env vars). |
Quick checklist
- Ensure Playwright browsers exist
npm run playwright:install
- Ensure build artifacts are fresh
- Full build + smoke:
npm run smoke - Quick smoke (assumes you already built):
npm run test:smoke:quick
- Run one smoke test first
- By name:
npm run test:smoke:quick -- -t "<substring>"
- By file:
npm run test:smoke:quick -- smoke/tests/<suite>.smoke.spec.ts
Common failure modes
“Binary not found”
- Smoke resolves the binary from
target/release/lotarby default. - Fix: run
npm run build, or setLOTAR_BINARY_PATH(orLOTAR_BIN) to a custom path.
- Smoke resolves the binary from
“Port already in use”
- The harness normally auto-picks a free port; failures may indicate a stuck server.
- Re-run with a single test in-band (
--runInBand) so it’s easier to spot lifecycle issues.
“SSE readiness / flaky waits”
- Smoke uses
LOTAR_SSE_READYhooks and server heartbeats; seedocs/help/serve.mdfor the testing aids.
- Smoke uses
Useful env vars
LOTAR_BINARY_PATH/LOTAR_BIN: override the binary used by smoke.LOTAR_TASKS_DIR,LOTAR_HOME: smoke sets these per-test (seesmoke/helpers/workspace.ts).RUST_LOG=debugorLOTAR_DEBUG=1: can help when diagnosing server/CLI behavior (keep logs free of secrets/PII).
Debugging approach
- Prefer
npx vitest watch --config smoke/vitest.config.ts --runInBandfor a tight loop. - If needed, temporarily enable inherited stdio in the smoke helpers while debugging (but keep changes scoped and revert before finalizing).