face-alignment

therealityreport's avatarfrom therealityreport

Face alignment, landmark extraction, and quality gating. Use when debugging alignment issues, reviewing rejected faces, or tuning quality thresholds.

⭐0stars🔀0forks📁View on GitHub🕐Updated Dec 30, 2025⬇️1downloads
[Debugging]

When & Why to Use This Skill

This Claude skill provides a comprehensive suite for face alignment, landmark extraction, and quality gating, specifically designed to help developers debug alignment artifacts and optimize embedding gating behavior in computer vision pipelines. It leverages advanced models like FAN for 68-point landmark extraction and offers diagnostic tools to resolve identity fragmentation and quality threshold issues.

Use Cases

  • Investigating and resolving face rejection issues by analyzing heuristic alignment quality scores and diagnostic JSON outputs.
  • Fine-tuning 'alignment_quality' thresholds in pipeline configurations to balance data precision and recall in facial recognition systems.
  • Extracting precise 68-point 2D landmarks and generating standardized face crops for downstream machine learning tasks.
  • Diagnosing landmark jitter and temporal instability in video sequences to improve the consistency of face tracking and embedding generation.
nameface-alignment
descriptionFace alignment, landmark extraction, and quality gating. Use when debugging alignment issues, reviewing rejected faces, or tuning quality thresholds.

Face Alignment Skill

Use this skill to debug face alignment artifacts and embedding gating behavior.

When to Use

  • Faces being rejected but you don't know why
  • Identity fragmentation that might be alignment-related
  • Need to adjust alignment quality thresholds
  • Debugging how alignment_quality is computed and consumed
  • Planning promotion/integration (FEATURE sandbox → production)

Sub-agents

Sub-agent Purpose
FaceAlign2DSubagent FAN 68-point 2D landmark extraction
FaceAlign3DSubagent 3DDFA_V2 dense 3D alignment (planned)
AlignmentQualitySubagent LUVLi-style uncertainty scoring (planned; heuristic today)

Key Skills

Run FAN alignment on a face bbox

Run alignment on detected face boxes (produces 68-point landmarks; optional aligned crop).

from FEATURES.face_alignment.src.run_fan_alignment import FANAligner, align_face_crop

aligner = FANAligner(model_type="2d", landmarks_type="2D", device="cpu")
landmarks = aligner.align_face(image, bbox)  # 68 x 2
aligned_crop = align_face_crop(image, landmarks, output_size=112)

Compute alignment_quality (heuristic today)

Get a per-face quality score (0–1). Current implementation is heuristic-based.

from FEATURES.face_alignment.src.alignment_quality import compute_alignment_quality

quality = compute_alignment_quality(bbox=bbox, landmarks_68=landmarks)

Generate episode artifacts

python -m FEATURES.face_alignment --episode-id <EP_ID>

Config Reference

Alignment Artifact Config: config/pipeline/face_alignment.yaml

Key Default Description
face_alignment.enabled true Produce face_alignment/aligned_faces.jsonl
face_alignment.model.type 2d FAN model variant (2d / 3d)
face_alignment.processing.stride 1 Sample every Nth frame
face_alignment.processing.batch_size 16 Faces per batch
face_alignment.processing.device auto auto / cuda / cpu

Embedding Gating Config: config/pipeline/embedding.yaml

Key Default Description
face_alignment.enabled true Enable/disable gating logic
face_alignment.min_alignment_quality 0.3 Skip faces below this threshold

Common Issues

Faces unexpectedly skipped before embedding

Cause: Face below quality threshold

Check: alignment_quality in data/manifests/{ep_id}/face_alignment/aligned_faces.jsonl

Fix: Lower threshold in config/pipeline/embedding.yaml:

face_alignment:
  min_alignment_quality: 0.2  # default is 0.3

High landmark_jitter

Cause: Unstable landmarks across frames

Check: Per-frame landmark variance in track

Fix:

  • Ensure FAN model loaded correctly
  • Check input face crop quality
  • May need temporal smoothing

3D head pose / profile gating isn’t available

Cause: 3DDFA_V2 isn’t implemented yet (planned).

Diagnostic Output

{
  "face_id": "F_42_100",
  "alignment_quality": 0.72,
  "landmarks_detected": 68,
  "head_pose": {
    "yaw": -25.5,
    "pitch": 10.2,
    "roll": 3.1
  },
  "quality_breakdown": {
    "eyes": 0.85,
    "nose": 0.80,
    "mouth": 0.65,
    "chin": 0.55
  },
  "aligner_used": "fan_2d"
}

Key Files

File Purpose
FEATURES/face_alignment/src/run_fan_alignment.py FAN landmarks + crop utilities
FEATURES/face_alignment/src/face_alignment_runner.py Episode runner (python -m FEATURES.face_alignment)
FEATURES/face_alignment/src/alignment_quality.py Heuristic alignment_quality
FEATURES/face_alignment/src/run_luvli_quality.py LUVLi-style scaffolding (not true LUVLi yet)
config/pipeline/face_alignment.yaml Alignment artifact config
config/pipeline/embedding.yaml Embedding gating config
FEATURES/face_alignment/tests/test_face_alignment.py Unit tests (synthetic)
tests/integration/test_face_alignment_pipeline.py Pipeline integration helpers/tests