memory-hygiene
Maintains memory cleanliness with deduplication, validation, and expiration
When & Why to Use This Skill
The Memory Hygiene skill is a specialized utility designed to maintain the integrity, performance, and accuracy of AI memory systems. By automating deduplication, schema validation, and stale data expiration, it prevents memory bloat and ensures that stored information remains high-quality and easily queryable for long-term agentic workflows.
Use Cases
- Optimizing performance: Compacting memory files exceeding 5MB or 1000+ entries to maintain fast query response times.
- Data Integrity: Automatically validating memory entries against a JSON schema before write operations to prevent data corruption.
- Automated Maintenance: Setting up periodic background tasks to expire and archive stale information based on TTL (Time To Live) timestamps.
- Bulk Import Cleanup: Merging duplicate or highly similar entries (over 80% similarity) after large-scale data migrations or repository indexing.
- Conflict Resolution: Maintaining the highest confidence versions of information when multiple sources provide overlapping data.
| name | memory-hygiene |
|---|---|
| description | Maintains memory cleanliness with deduplication, validation, and expiration |
| version | 1.0.0 |
| author | Claude Memory System |
| tags | [memory, maintenance, validation, deduplication] |
Memory Hygiene Skill
Purpose
Ensure memory remains queryable, compact, and consistent through automated hygiene operations. Prevents memory bloat and maintains data quality.
When to Use
- Before bulk memory operations (imports, migrations)
- When memory file exceeds 5MB or 1000+ patterns
- Periodic maintenance (daily/weekly background task)
- After detecting duplicate or stale entries
- When schema validation fails
Operations
1. Compact Memory
Trigger: Memory file >5MB or >1000 patterns Process:
- Read all memory entries
- Deduplicate identical patterns
- Merge similar entries (>80% similarity)
- Bucket by topic for faster queries
- Write compacted version with backup
Script: scripts/compact_memory.py
Output: Compression report with stats
2. Validate Schema
Trigger: Before ANY memory write operation Process:
- Load schema from
assets/schema_v1.json - Check all required fields present
- Validate data types and constraints
- Check TTL format (ISO8601)
- Verify confidence score (0.0-1.0)
Script: scripts/validate_schema.py
Output: Boolean pass/fail + error details
3. Expire Stale Entries
Trigger: Daily background task or manual Process:
- Scan all entries for TTL field
- Compare TTL to current timestamp
- Archive expired entries (if configured)
- Remove from active memory
- Log expired entry IDs
Script: scripts/expire_stale.py
Output: List of expired entries + archive path
4. Merge Duplicates
Trigger: After bulk imports or weekly maintenance Process:
- Group entries by topic + scope
- Calculate similarity scores (fuzzy matching)
- Merge entries with >80% similarity
- Keep highest confidence version
- Preserve metadata from all sources
Script: scripts/merge_duplicates.py
Output: Deduplication report
Schema Definition
All memory entries MUST conform to:
{
"topic": "string (required, max 100 chars)",
"scope": "global|repository (required)",
"value": "any (required)",
"TTL": "ISO8601 timestamp (optional)",
"source": "user|agent|system (required)",
"confidence": "0.0-1.0 (required)",
"metadata": {
"repository": "string (optional)",
"created": "ISO8601 (auto-generated)",
"updated": "ISO8601 (auto-generated)"
}
}
See references/schema.md for complete specification.
Integration with Memory Tool
Before Write Operations
# Always validate before writing
validation = execute_skill('memory-hygiene', {
'operation': 'validate',
'data': memory_entry
})
if validation['valid']:
write_to_memory(memory_entry)
else:
handle_validation_errors(validation['errors'])
Periodic Maintenance
# Daily cron job
0 2 * * * node ~/.claude/memory/skill-executor.js execute memory-hygiene '{"operation":"expire"}'
# Weekly compaction
0 3 * * 0 node ~/.claude/memory/skill-executor.js execute memory-hygiene '{"operation":"compact"}'
After Bulk Operations
# After importing many entries
import_entries(bulk_data)
# Immediately compact and deduplicate
execute_skill('memory-hygiene', {
'operation': 'compact_and_merge'
})
Safety Measures
All hygiene operations:
- Create backup before modification (
/memories/.backups/) - Log all changes to
~/.claude/memory/hygiene.log - Dry-run mode available with
--previewflag - Atomic operations - rollback on failure
- Preserve provenance - track all transformations
Examples
Good Memory Entries
See references/examples.md for examples of well-formed entries.
Bad Memory Entries
See references/examples.md for anti-patterns to avoid.
Performance
- Validation: <10ms per entry
- Compaction: ~100ms per 1000 entries
- Deduplication: ~200ms per 1000 entries
- Expiration: ~50ms per 1000 entries
Error Handling
- Invalid schema: Returns detailed error with field name
- Corrupted data: Archives problematic entries separately
- Disk full: Fails gracefully with clear error
- Backup failure: Aborts operation, preserves original
Configuration
Edit assets/hygiene_config.json to customize:
- Similarity threshold for merging (default: 0.8)
- Archive location (default:
/memories/.archive/) - Backup retention (default: 7 days)
- Max memory size before auto-compact (default: 5MB)
Integration Points
Memory Tool Operations
insert→ validate_schema()replace→ validate_schema()delete→ log_deletion()rename→ update_metadata()
Context Editing
After context clearing, run compaction to optimize memory.
Background Tasks
Set up cron jobs for automated maintenance (see Integration section).
Memory Hygiene Skill v1.0.0 - Keeping your memory clean and queryable