evaluate-model
Measure model performance on test datasets. Use when assessing accuracy, precision, recall, and other metrics.
When & Why to Use This Skill
The evaluate-model skill provides a robust framework for measuring machine learning model performance across classification and regression tasks. It automates the calculation of essential metrics like accuracy, precision, recall, and MSE, enabling developers to validate model reliability, detect overfitting, and benchmark different architectures against test datasets for optimized AI performance.
Use Cases
- Benchmarking multiple model architectures to determine the most accurate solution for a specific dataset.
- Detecting overfitting or underfitting by comparing performance metrics between training and validation sets.
- Generating detailed evaluation reports including confusion matrices and error analysis for classification models.
- Standardizing performance reporting for technical documentation and research papers using Mojo-based ML patterns.
| name | evaluate-model |
|---|---|
| description | "Measure model performance on test datasets. Use when assessing accuracy, precision, recall, and other metrics." |
| mcp_fallback | none |
| category | ml |
| tier | 2 |
| user-invocable | false |
Evaluate Model
Measure machine learning model performance using appropriate metrics for the task (classification, regression, etc.).
When to Use
- Comparing different model architectures
- Assessing performance on test/validation datasets
- Detecting overfitting or underfitting
- Reporting model accuracy for papers and documentation
Quick Reference
# Mojo model evaluation pattern
struct ModelEvaluator:
fn evaluate_classification(
mut self,
predictions: ExTensor,
ground_truth: ExTensor
) -> Tuple[Float32, Float32, Float32]:
# Returns accuracy, precision, recall
...
fn evaluate_regression(
mut self,
predictions: ExTensor,
ground_truth: ExTensor
) -> Tuple[Float32, Float32]:
# Returns MSE, MAE
...
Workflow
- Load test data: Prepare test/validation dataset
- Generate predictions: Run model inference on test set
- Select metrics: Choose appropriate metrics (accuracy, precision, recall, F1, AUC, MSE, etc.)
- Calculate metrics: Compute performance metrics
- Analyze results: Compare to baseline and identify strengths/weaknesses
Output Format
Evaluation report:
- Task type (classification, regression, etc.)
- Metrics (accuracy, precision, recall, F1, AUC, etc.)
- Per-class breakdown (if applicable)
- Comparison to baseline model
- Confusion matrix (classification)
- Error analysis
References
- See CLAUDE.md > Language Preference (Mojo for ML models)
- See
train-modelskill for model training - See
/notes/review/mojo-ml-patterns.mdfor Mojo tensor operations