model-compare

rawwerks 的头像作者 rawwerks

使用布尔操作(IoU、Dice、精确度/召回率)比较3D CAD模型。在评估生成的模型与黄金参考的差异、比较CAD修订版本或计算ML训练的相似度度量时使用。在模型差异、比较模型、IoU、交并比、模型相似度、CAD比较、STEP差异、3D评估、黄金参考、生成模型、精确度召回率3D时触发。

📖 中文导读:使用布尔操作(IoU、Dice、精确度/召回率)比较3D CAD模型。在评估生成的模型与黄金参考的差异、比较CAD修订版本或计算ML训练的相似度度量时使用。在模型差异、比较模型、IoU、交并比、模型相似度、CAD比较、STEP差异、3D评估、黄金参考、生成模型、精确度召回率3D时触发。 以下正文为该技能的原始 SKILL.md 内容(英文),上方安装命令可直接使用。

name: model-compare description: Compare 3D CAD models using boolean operations (IoU, Dice, precision/recall). Use when evaluating generated models against gold references, diffing CAD revisions, or computing similarity metrics for ML training. Triggers on: model diff, compare models, IoU, intersection over union, model similarity, CAD comparison, STEP diff, 3D evaluation, gold reference, generated model, precision recall 3D.

3D Model Comparison Tool

Compare CAD models using boolean operations to compute similarity metrics like IoU, Dice, precision, and recall. Useful for:

  • Evaluating ML-generated models against gold references
  • Comparing revisions of CAD designs
  • Computing metrics for training 3D generative models
  • Visualizing geometric differences

Quick Start

# Compare two STEP files
uvx --from build123d python scripts/model_diff.py reference.step generated.step

# JSON output for training pipelines
uvx --from build123d python scripts/model_diff.py ref.step gen.step --json --no-export

# Demo mode (no files needed)
uvx --from build123d python scripts/model_diff.py --demo

Supported Formats

Format Extension Notes
STEP .step, .stp Recommended - full CAD fidelity
BREP .brep OpenCASCADE native format
STL .stl Mesh format - may have boolean issues

Output Metrics

Primary Metrics (for ML training)

Metric Range Description
IoU (Jaccard) 0-1 `
Dice (F1) 0-1 `2
Precision 0-1 `
Recall 0-1 `

Diagnostic Metrics

Metric Description
volume_ratio B/A volume ratio (1.0 = same size)
center_offset Distance between centers of mass
bbox_iou Bounding box IoU (coarse alignment)
size_ratio_x/y/z Per-axis scale comparison
surface_ratio Surface area comparison

Interpretation

The tool provides automatic interpretation:

  • Over-generating: Low precision, high extra geometry
  • Under-generating: Low recall, missing geometry
  • Size issues: Volume ratio far from 1.0
  • Position issues: Large center offset

CLI Options

usage: model_diff.py [-h] [-o OUTPUT_DIR] [--json] [--no-export] [--demo]
                     [reference] [generated]

positional arguments:
  reference          Reference/gold model file (STEP, BREP, or STL)
  generated          Generated/predicted model file to compare

options:
  -o, --output-dir   Output directory for GLB files (default: .)
  --json             Output only JSON metrics (for pipelines)
  --no-export        Skip exporting GLB visualization files
  --demo             Run with built-in demo models

Output Files

When --no-export is not set, produces GLB files for visualization:

File Description
diff_reference.glb The reference model (A)
diff_generated.glb The generated model (B)
diff_missing.glb Geometry in A but not B (under-generation)
diff_extra.glb Geometry in B but not A (over-generation)
diff_common.glb Geometry in both (correct match)

Example: Training Pipeline Integration

# Batch evaluation
for gen in outputs/*.step; do
    uvx --from build123d python model_diff.py gold.step "$gen" --json --no-export
done | jq -s '{
    avg_iou: (map(.iou) | add / length),
    avg_precision: (map(.precision) | add / length),
    avg_recall: (map(.recall) | add / length)
}'

Example: Loss Function

# In your training code, use metrics for loss:
loss = (
    (1 - metrics['iou']) * 1.0 +           # Primary shape match
    abs(1 - metrics['volume_ratio']) * 0.5 + # Scale accuracy
    metrics['center_offset'] * 0.1           # Position accuracy
)

How It Works

The tool uses boolean operations from OpenCASCADE (via build123d):

Missing  = Reference - Generated  (A - B)
Extra    = Generated - Reference  (B - A)
Common   = Reference & Generated  (A ∩ B)
Union    = Reference + Generated  (A ∪ B)

IoU      = volume(Common) / volume(Union)
Dice     = 2 * volume(Common) / (volume(A) + volume(B))
Precision = volume(Common) / volume(B)
Recall    = volume(Common) / volume(A)

Sample Output

=================================================================
  3D MODEL COMPARISON REPORT
  Reference (A) vs Generated (B)
=================================================================

──────────────────────────── VOLUMES ────────────────────────────
  Reference (A):          51,433.629
  Generated (B):          45,904.426
  Intersection (A∩B):     42,292.031
  Missing (A-B):           9,141.598  (17.8% of A)
  Extra (B-A):             3,612.395  (7.9% of B)

──────────────────────── PRIMARY METRICS ────────────────────────
  IoU (Jaccard):              0.7683  (1.0 = identical)
  Dice (F1):                  0.8690  (1.0 = identical)
  Precision:                  0.9213  (correctness of B)
  Recall:                     0.8223  (coverage of A)

──────────────────────── INTERPRETATION ─────────────────────────
  △ Partial match (IoU > 50%)
  → Under-generating: 17.8% of A is missing
  → Undersized by 10.8%
=================================================================