ai-engineering
Building production AI applications with Foundation Models. Covers prompt engineering, RAG, agents, finetuning, evaluation, and deployment. Use when working with LLMs, building AI features, or architecting AI systems.
When & Why to Use This Skill
This Claude skill provides a comprehensive framework for building production-grade AI applications using Foundation Models. It covers the entire AI engineering lifecycle—from prompt engineering and RAG systems to model finetuning, evaluation, and deployment—enabling developers to architect scalable, reliable, and high-performance AI systems.
Use Cases
- Architecting RAG Systems: Designing efficient data chunking, embedding, and retrieval pipelines to integrate external knowledge bases with LLMs.
- Developing AI Agents: Implementing multi-step reasoning, planning strategies, and tool-use capabilities for complex autonomous workflows.
- System Evaluation & Benchmarking: Establishing rigorous evaluation pipelines using AI-as-a-judge and comparative metrics to ensure output quality and reliability.
- Production Optimization: Reducing latency and costs through inference optimization techniques like quantization, caching, and speculative decoding.
- Safety & Guardrails: Implementing input/output filters and PII protection to ensure AI applications meet security and compliance standards.
| name | ai-engineering |
|---|---|
| description | Building production AI applications with Foundation Models. Covers prompt engineering, RAG, agents, finetuning, evaluation, and deployment. Use when working with LLMs, building AI features, or architecting AI systems. |
AI Engineering Skills
Comprehensive skills for building AI applications with Foundation Models.
AI Engineering Stack
┌─────────────────────────────────────────────────────┐
│ APPLICATION LAYER │
│ Prompt Engineering, RAG, Agents, Guardrails │
├─────────────────────────────────────────────────────┤
│ MODEL LAYER │
│ Model Selection, Finetuning, Evaluation │
├─────────────────────────────────────────────────────┤
│ INFRASTRUCTURE LAYER │
│ Inference Optimization, Caching, Orchestration │
└─────────────────────────────────────────────────────┘
12 Core Skills
| Skill | Description | Guide |
|---|---|---|
| Foundation Models | Model architecture, sampling, structured outputs | foundation-models/ |
| Evaluation Methodology | Metrics, AI-as-judge, comparative evaluation | evaluation-methodology/ |
| AI System Evaluation | End-to-end evaluation, benchmarks, model selection | ai-system-evaluation/ |
| Prompt Engineering | System prompts, few-shot, chain-of-thought, defense | prompt-engineering/ |
| RAG Systems | Chunking, embedding, retrieval, reranking | rag-systems/ |
| AI Agents | Tool use, planning strategies, memory systems | ai-agents/ |
| Finetuning | LoRA, QLoRA, PEFT, model merging | finetuning/ |
| Dataset Engineering | Data quality, curation, synthesis, annotation | dataset-engineering/ |
| Inference Optimization | Quantization, batching, caching, speculative decoding | inference-optimization/ |
| AI Architecture | Gateway, routing, observability, deployment | ai-architecture/ |
| Guardrails & Safety | Input/output guards, PII protection, injection defense | guardrails-safety/ |
| User Feedback | Explicit/implicit signals, feedback loops, A/B testing | user-feedback/ |
Development Process
1. Use Case Evaluation → 2. Model Selection → 3. Evaluation Pipeline
↓
4. Prompt Engineering → 5. Context (RAG/Agents) → 6. Finetuning (if needed)
↓
7. Inference Optimization → 8. Deployment → 9. Monitoring & Feedback
Quick Decision Guide
| Need | Start With |
|---|---|
| Improve output quality | prompt-engineering |
| Add external knowledge | rag-systems |
| Multi-step reasoning | ai-agents |
| Reduce latency/cost | inference-optimization |
| Measure quality | evaluation-methodology |
| Protect system | guardrails-safety |
Reference
Based on "AI Engineering" by Chip Huyen (O'Reilly, 2025).