memory-hygiene
Maintains memory cleanliness with deduplication, validation, and expiration
When & Why to Use This Skill
The Memory Hygiene skill is a specialized utility designed to optimize AI agent performance by maintaining a clean, structured, and efficient long-term memory system. It automates critical maintenance tasks—including deduplication, schema validation, and stale data expiration—to prevent memory bloat, reduce noise, and ensure high-quality context retrieval for more accurate agent responses.
Use Cases
- Automated System Maintenance: Periodically compact memory files and archive expired entries via cron jobs to keep the agent's knowledge base lean and responsive.
- Data Integrity Enforcement: Use real-time schema validation before write operations to ensure all stored information follows strict formatting and confidence requirements.
- Post-Import Optimization: Automatically merge similar entries and remove redundant patterns after performing bulk data migrations or repository indexing.
- Token and Cost Efficiency: Reduce context window overhead by deduplicating information and merging high-similarity entries, leading to faster processing and lower API costs.
| name | memory-hygiene |
|---|---|
| description | Maintains memory cleanliness with deduplication, validation, and expiration |
| version | 1.0.0 |
| author | Claude Memory System |
| tags | [memory, maintenance, validation, deduplication] |
Memory Hygiene Skill
Purpose
Ensure memory remains queryable, compact, and consistent through automated hygiene operations. Prevents memory bloat and maintains data quality.
When to Use
- Before bulk memory operations (imports, migrations)
- When memory file exceeds 5MB or 1000+ patterns
- Periodic maintenance (daily/weekly background task)
- After detecting duplicate or stale entries
- When schema validation fails
Operations
1. Compact Memory
Trigger: Memory file >5MB or >1000 patterns Process:
- Read all memory entries
- Deduplicate identical patterns
- Merge similar entries (>80% similarity)
- Bucket by topic for faster queries
- Write compacted version with backup
Script: scripts/compact_memory.py
Output: Compression report with stats
2. Validate Schema
Trigger: Before ANY memory write operation Process:
- Load schema from
assets/schema_v1.json - Check all required fields present
- Validate data types and constraints
- Check TTL format (ISO8601)
- Verify confidence score (0.0-1.0)
Script: scripts/validate_schema.py
Output: Boolean pass/fail + error details
3. Expire Stale Entries
Trigger: Daily background task or manual Process:
- Scan all entries for TTL field
- Compare TTL to current timestamp
- Archive expired entries (if configured)
- Remove from active memory
- Log expired entry IDs
Script: scripts/expire_stale.py
Output: List of expired entries + archive path
4. Merge Duplicates
Trigger: After bulk imports or weekly maintenance Process:
- Group entries by topic + scope
- Calculate similarity scores (fuzzy matching)
- Merge entries with >80% similarity
- Keep highest confidence version
- Preserve metadata from all sources
Script: scripts/merge_duplicates.py
Output: Deduplication report
Schema Definition
All memory entries MUST conform to:
{
"topic": "string (required, max 100 chars)",
"scope": "global|repository (required)",
"value": "any (required)",
"TTL": "ISO8601 timestamp (optional)",
"source": "user|agent|system (required)",
"confidence": "0.0-1.0 (required)",
"metadata": {
"repository": "string (optional)",
"created": "ISO8601 (auto-generated)",
"updated": "ISO8601 (auto-generated)"
}
}
See references/schema.md for complete specification.
Integration with Memory Tool
Before Write Operations
# Always validate before writing
validation = execute_skill('memory-hygiene', {
'operation': 'validate',
'data': memory_entry
})
if validation['valid']:
write_to_memory(memory_entry)
else:
handle_validation_errors(validation['errors'])
Periodic Maintenance
# Daily cron job
0 2 * * * node ~/.claude/memory/skill-executor.js execute memory-hygiene '{"operation":"expire"}'
# Weekly compaction
0 3 * * 0 node ~/.claude/memory/skill-executor.js execute memory-hygiene '{"operation":"compact"}'
After Bulk Operations
# After importing many entries
import_entries(bulk_data)
# Immediately compact and deduplicate
execute_skill('memory-hygiene', {
'operation': 'compact_and_merge'
})
Safety Measures
All hygiene operations:
- Create backup before modification (
/memories/.backups/) - Log all changes to
~/.claude/memory/hygiene.log - Dry-run mode available with
--previewflag - Atomic operations - rollback on failure
- Preserve provenance - track all transformations
Examples
Good Memory Entries
See references/examples.md for examples of well-formed entries.
Bad Memory Entries
See references/examples.md for anti-patterns to avoid.
Performance
- Validation: <10ms per entry
- Compaction: ~100ms per 1000 entries
- Deduplication: ~200ms per 1000 entries
- Expiration: ~50ms per 1000 entries
Error Handling
- Invalid schema: Returns detailed error with field name
- Corrupted data: Archives problematic entries separately
- Disk full: Fails gracefully with clear error
- Backup failure: Aborts operation, preserves original
Configuration
Edit assets/hygiene_config.json to customize:
- Similarity threshold for merging (default: 0.8)
- Archive location (default:
/memories/.archive/) - Backup retention (default: 7 days)
- Max memory size before auto-compact (default: 5MB)
Integration Points
Memory Tool Operations
insert→ validate_schema()replace→ validate_schema()delete→ log_deletion()rename→ update_metadata()
Context Editing
After context clearing, run compaction to optimize memory.
Background Tasks
Set up cron jobs for automated maintenance (see Integration section).
Memory Hygiene Skill v1.0.0 - Keeping your memory clean and queryable