data-processor
Process and transform arrays of data with common operations like filtering, mapping, and aggregation
When & Why to Use This Skill
The Data Processor skill is a high-performance utility designed for efficient array manipulation and data transformation. By leveraging code execution instead of natural language descriptions, it enables Claude to filter, map, aggregate, and sort large datasets with significant token savings and high precision. It is ideal for developers and data analysts who need to handle hundreds or thousands of records without exhausting context windows.
Use Cases
- Cleaning and normalizing customer contact lists by removing duplicates and formatting fields (e.g., trimming whitespace, lowercase emails).
- Summarizing financial transactions or sales data to calculate total revenue, averages, and counts across specific categories.
- Filtering large JSON datasets from API responses to extract only relevant records based on complex, multi-step business logic.
- Preparing raw data for reporting by sorting entries chronologically and merging multiple datasets into a unified, structured format.
- Reducing token usage in long-running conversations by processing data programmatically rather than describing every transformation step in text.
| name | data-processor |
|---|---|
| description | Process and transform arrays of data with common operations like filtering, mapping, and aggregation |
| version | 1.0.0 |
Data Processor Skill
A general-purpose data processing skill for transforming arrays of objects. This skill demonstrates the token efficiency benefits of code execution - instead of describing transformations in natural language, write code once and reuse it.
What This Skill Does
Processes arrays of data with common transformations:
- Filter records based on conditions
- Map fields to new values
- Aggregate data (sum, average, count, etc.)
- Sort and group data
- Remove duplicates
- Merge datasets
When to Use This Skill
Use this skill when you need to:
- Transform large datasets (hundreds or thousands of records)
- Apply consistent business logic to data
- Aggregate or summarize data
- Clean or normalize data
- Combine data from multiple sources
Token Efficiency: Processing 1000 records in code uses ~500 tokens. Describing the same operations in natural language would use ~50,000 tokens.
Implementation
/**
* Data Processor - General purpose data transformation
* @param {Array} data - Array of objects to process
* @param {Object} operations - Operations to apply
* @returns {Object} Processed data and statistics
*/
async function processData(data, operations = {}) {
if (!Array.isArray(data)) {
throw new Error('Data must be an array');
}
let result = [...data];
const stats = {
inputCount: data.length,
operations: [],
};
// Filter operation
if (operations.filter) {
const beforeCount = result.length;
result = result.filter(operations.filter);
stats.operations.push({
type: 'filter',
recordsRemoved: beforeCount - result.length
});
}
// Map operation (transform fields)
if (operations.map) {
result = result.map(operations.map);
stats.operations.push({ type: 'map' });
}
// Sort operation
if (operations.sort) {
const { field, order = 'asc' } = operations.sort;
result.sort((a, b) => {
const aVal = a[field];
const bVal = b[field];
const comparison = aVal < bVal ? -1 : aVal > bVal ? 1 : 0;
return order === 'asc' ? comparison : -comparison;
});
stats.operations.push({ type: 'sort', field, order });
}
// Aggregate operation
if (operations.aggregate) {
const { field, operation: aggOp } = operations.aggregate;
const values = result.map(r => r[field]).filter(v => v != null);
let aggregateResult;
switch (aggOp) {
case 'sum':
aggregateResult = values.reduce((sum, v) => sum + v, 0);
break;
case 'average':
aggregateResult = values.reduce((sum, v) => sum + v, 0) / values.length;
break;
case 'count':
aggregateResult = values.length;
break;
case 'min':
aggregateResult = Math.min(...values);
break;
case 'max':
aggregateResult = Math.max(...values);
break;
default:
throw new Error(`Unknown aggregate operation: ${aggOp}`);
}
stats.aggregateResult = {
field,
operation: aggOp,
value: aggregateResult
};
}
// Remove duplicates
if (operations.unique) {
const { field } = operations.unique;
const seen = new Set();
const beforeCount = result.length;
result = result.filter(item => {
const key = item[field];
if (seen.has(key)) return false;
seen.add(key);
return true;
});
stats.operations.push({
type: 'unique',
field,
duplicatesRemoved: beforeCount - result.length
});
}
stats.outputCount = result.length;
return {
data: result,
stats
};
}
module.exports = processData;
Examples
Example 1: Filter and Sort
const processData = require('/skills/data-processor.js');
const salesData = [
{ id: 1, amount: 150, status: 'completed' },
{ id: 2, amount: 200, status: 'pending' },
{ id: 3, amount: 175, status: 'completed' },
{ id: 4, amount: 225, status: 'completed' }
];
const result = await processData(salesData, {
filter: (record) => record.status === 'completed',
sort: { field: 'amount', order: 'desc' }
});
console.log(result);
// Output:
// {
// data: [
// { id: 4, amount: 225, status: 'completed' },
// { id: 3, amount: 175, status: 'completed' },
// { id: 1, amount: 150, status: 'completed' }
// ],
// stats: {
// inputCount: 4,
// operations: [
// { type: 'filter', recordsRemoved: 1 },
// { type: 'sort', field: 'amount', order: 'desc' }
// ],
// outputCount: 3
// }
// }
Example 2: Aggregate Data
const processData = require('/skills/data-processor.js');
const orders = [
{ orderId: 1, total: 100 },
{ orderId: 2, total: 150 },
{ orderId: 3, total: 200 }
];
const result = await processData(orders, {
aggregate: { field: 'total', operation: 'sum' }
});
console.log(result.stats.aggregateResult);
// Output: { field: 'total', operation: 'sum', value: 450 }
Example 3: Complex Transformation
const processData = require('/skills/data-processor.js');
const customers = [
{ name: ' John Doe ', email: 'JOHN@EXAMPLE.COM', age: 30 },
{ name: 'Jane Smith', email: 'jane@example.com', age: 25 },
{ name: ' John Doe ', email: 'JOHN@EXAMPLE.COM', age: 30 } // duplicate
];
const result = await processData(customers, {
map: (customer) => ({
name: customer.name.trim(),
email: customer.email.toLowerCase(),
age: customer.age
}),
unique: { field: 'email' },
filter: (customer) => customer.age >= 25,
sort: { field: 'age', order: 'asc' }
});
console.log(result.data);
// Output:
// [
// { name: 'Jane Smith', email: 'jane@example.com', age: 25 },
// { name: 'John Doe', email: 'john@example.com', age: 30 }
// ]
Integration with MCP Tools
This skill works great in combination with MCP tools:
// Fetch data from an MCP tool
const rawData = await callMCPTool('database__query', {
query: 'SELECT * FROM customers WHERE created_date > "2024-01-01"'
});
// Process with the skill
const processData = require('/skills/data-processor.js');
const result = await processData(rawData, {
filter: (r) => r.status === 'active',
sort: { field: 'revenue', order: 'desc' },
aggregate: { field: 'revenue', operation: 'sum' }
});
// Save results
await callMCPTool('storage__save', {
key: 'processed_customers',
value: result.data
});
// Return summary to agent (not full data)
return {
processedRecords: result.stats.outputCount,
totalRevenue: result.stats.aggregateResult.value
};
Tips and Best Practices
- Save Intermediate Results: For large datasets, save to
/workspaceafter each major operation - Return Summaries: Send statistics to the agent, not full datasets
- Chain Operations: Combine multiple operations for complex transformations
- Validate Input: Always check data types and handle edge cases
- Reuse This Skill: Save to
/skillsand use across multiple tasks
Related Skills
validator- Validate data before processingexporter- Export processed data to various formatsaggregator- Advanced statistical aggregations
Performance Notes
This skill can process:
- 1,000 records: < 50ms
- 10,000 records: < 200ms
- 100,000 records: < 2s
All operations use efficient JavaScript array methods with O(n) or O(n log n) complexity.
Inspired by: The Anthropic skills pattern for token-efficient data processing. See Code Execution with MCP for the philosophy behind this approach.