llm-application-dev
Building applications with Large Language Models - prompt engineering, RAG patterns, and LLM integration. Use for AI-powered features, chatbots, or LLM-based automation.
When & Why to Use This Skill
This Claude skill serves as a comprehensive framework for Large Language Model (LLM) application development. It provides production-ready patterns for prompt engineering (including Few-Shot and Chain of Thought), Retrieval-Augmented Generation (RAG) pipelines, and seamless API integration with providers like OpenAI and Anthropic. By offering structured code for document chunking, vector storage, and robust error handling, it enables developers to build scalable, AI-powered features and automated workflows with industry best practices.
Use Cases
- 1. AI Chatbot Development: Implementing structured prompts and RAG patterns to create context-aware conversational agents for customer support or internal use.
- 2. Enterprise Knowledge Retrieval: Building systems that query private documentation by combining vector databases (like Supabase/pgvector) with LLM reasoning.
- 3. Complex Data Classification: Utilizing few-shot prompting and chain-of-thought techniques to automate sentiment analysis and structured data extraction.
- 4. Production-Grade AI Integration: Deploying resilient LLM services with built-in exponential backoff, token management, and streaming responses for optimized user experience.
| name | llm-application-dev |
|---|---|
| description | Building applications with Large Language Models - prompt engineering, RAG patterns, and LLM integration. Use for AI-powered features, chatbots, or LLM-based automation. |
| source | wshobson/agents |
| license | MIT |
LLM Application Development
Prompt Engineering
Structured Prompts
const systemPrompt = `You are a helpful assistant that answers questions about our product.
RULES:
- Only answer questions about our product
- If you don't know, say "I don't know"
- Keep responses concise (under 100 words)
- Never make up information
CONTEXT:
{context}`;
const userPrompt = `Question: {question}`;
Few-Shot Examples
const prompt = `Classify the sentiment of customer feedback.
Examples:
Input: "Love this product!"
Output: positive
Input: "Worst purchase ever"
Output: negative
Input: "It works fine"
Output: neutral
Input: "${customerFeedback}"
Output:`;
Chain of Thought
const prompt = `Solve this step by step:
Question: ${question}
Let's think through this:
1. First, identify the key information
2. Then, determine the approach
3. Finally, calculate the answer
Step-by-step solution:`;
API Integration
OpenAI Pattern
import OpenAI from 'openai';
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
async function chat(messages: Message[]): Promise<string> {
const response = await openai.chat.completions.create({
model: 'gpt-4',
messages,
temperature: 0.7,
max_tokens: 500,
});
return response.choices[0].message.content ?? '';
}
Anthropic Pattern
import Anthropic from '@anthropic-ai/sdk';
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
async function chat(prompt: string): Promise<string> {
const response = await anthropic.messages.create({
model: 'claude-3-opus-20240229',
max_tokens: 1024,
messages: [{ role: 'user', content: prompt }],
});
return response.content[0].type === 'text'
? response.content[0].text
: '';
}
Streaming Responses
async function* streamChat(prompt: string) {
const stream = await openai.chat.completions.create({
model: 'gpt-4',
messages: [{ role: 'user', content: prompt }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content;
if (content) yield content;
}
}
RAG (Retrieval-Augmented Generation)
Basic RAG Pipeline
async function ragQuery(question: string): Promise<string> {
// 1. Embed the question
const questionEmbedding = await embedText(question);
// 2. Search vector database
const relevantDocs = await vectorDb.search(questionEmbedding, { limit: 5 });
// 3. Build context
const context = relevantDocs.map(d => d.content).join('\n\n');
// 4. Generate answer
const prompt = `Answer based on this context:\n${context}\n\nQuestion: ${question}`;
return await chat(prompt);
}
Document Chunking
function chunkDocument(text: string, options: ChunkOptions): string[] {
const { chunkSize = 1000, overlap = 200 } = options;
const chunks: string[] = [];
let start = 0;
while (start < text.length) {
const end = Math.min(start + chunkSize, text.length);
chunks.push(text.slice(start, end));
start += chunkSize - overlap;
}
return chunks;
}
Embedding Storage
// Using Supabase with pgvector
async function storeEmbeddings(docs: Document[]) {
for (const doc of docs) {
const embedding = await embedText(doc.content);
await supabase.from('documents').insert({
content: doc.content,
metadata: doc.metadata,
embedding: embedding, // vector column
});
}
}
async function searchSimilar(query: string, limit = 5) {
const embedding = await embedText(query);
const { data } = await supabase.rpc('match_documents', {
query_embedding: embedding,
match_count: limit,
});
return data;
}
Error Handling
async function safeLLMCall<T>(
fn: () => Promise<T>,
options: { retries?: number; fallback?: T }
): Promise<T> {
const { retries = 3, fallback } = options;
for (let i = 0; i < retries; i++) {
try {
return await fn();
} catch (error) {
if (error.status === 429) {
// Rate limit - exponential backoff
await sleep(Math.pow(2, i) * 1000);
continue;
}
if (i === retries - 1) {
if (fallback !== undefined) return fallback;
throw error;
}
}
}
throw new Error('Max retries exceeded');
}
Best Practices
- Token Management: Track usage and set limits
- Caching: Cache embeddings and common queries
- Evaluation: Test prompts with diverse inputs
- Guardrails: Validate outputs before using
- Logging: Log prompts and responses for debugging
- Cost Control: Use cheaper models for simple tasks
- Latency: Stream responses for better UX
- Privacy: Don't send PII to external APIs