memory-hygiene

Maintains memory cleanliness with deduplication, validation, and expiration

⭐5stars🔀1forks📁View on GitHub🕐Updated Jan 11, 2026

When & Why to Use This Skill

The Memory Hygiene skill is a specialized utility designed to optimize AI agent performance by maintaining a clean, structured, and efficient long-term memory system. It automates critical maintenance tasks—including deduplication, schema validation, and stale data expiration—to prevent memory bloat, reduce noise, and ensure high-quality context retrieval for more accurate agent responses.

Use Cases

  • Automated System Maintenance: Periodically compact memory files and archive expired entries via cron jobs to keep the agent's knowledge base lean and responsive.
  • Data Integrity Enforcement: Use real-time schema validation before write operations to ensure all stored information follows strict formatting and confidence requirements.
  • Post-Import Optimization: Automatically merge similar entries and remove redundant patterns after performing bulk data migrations or repository indexing.
  • Token and Cost Efficiency: Reduce context window overhead by deduplicating information and merging high-similarity entries, leading to faster processing and lower API costs.
namememory-hygiene
descriptionMaintains memory cleanliness with deduplication, validation, and expiration
version1.0.0
authorClaude Memory System
tags[memory, maintenance, validation, deduplication]

Memory Hygiene Skill

Purpose

Ensure memory remains queryable, compact, and consistent through automated hygiene operations. Prevents memory bloat and maintains data quality.

When to Use

  • Before bulk memory operations (imports, migrations)
  • When memory file exceeds 5MB or 1000+ patterns
  • Periodic maintenance (daily/weekly background task)
  • After detecting duplicate or stale entries
  • When schema validation fails

Operations

1. Compact Memory

Trigger: Memory file >5MB or >1000 patterns Process:

  1. Read all memory entries
  2. Deduplicate identical patterns
  3. Merge similar entries (>80% similarity)
  4. Bucket by topic for faster queries
  5. Write compacted version with backup

Script: scripts/compact_memory.py Output: Compression report with stats

2. Validate Schema

Trigger: Before ANY memory write operation Process:

  1. Load schema from assets/schema_v1.json
  2. Check all required fields present
  3. Validate data types and constraints
  4. Check TTL format (ISO8601)
  5. Verify confidence score (0.0-1.0)

Script: scripts/validate_schema.py Output: Boolean pass/fail + error details

3. Expire Stale Entries

Trigger: Daily background task or manual Process:

  1. Scan all entries for TTL field
  2. Compare TTL to current timestamp
  3. Archive expired entries (if configured)
  4. Remove from active memory
  5. Log expired entry IDs

Script: scripts/expire_stale.py Output: List of expired entries + archive path

4. Merge Duplicates

Trigger: After bulk imports or weekly maintenance Process:

  1. Group entries by topic + scope
  2. Calculate similarity scores (fuzzy matching)
  3. Merge entries with >80% similarity
  4. Keep highest confidence version
  5. Preserve metadata from all sources

Script: scripts/merge_duplicates.py Output: Deduplication report

Schema Definition

All memory entries MUST conform to:

{
  "topic": "string (required, max 100 chars)",
  "scope": "global|repository (required)",
  "value": "any (required)",
  "TTL": "ISO8601 timestamp (optional)",
  "source": "user|agent|system (required)",
  "confidence": "0.0-1.0 (required)",
  "metadata": {
    "repository": "string (optional)",
    "created": "ISO8601 (auto-generated)",
    "updated": "ISO8601 (auto-generated)"
  }
}

See references/schema.md for complete specification.

Integration with Memory Tool

Before Write Operations

# Always validate before writing
validation = execute_skill('memory-hygiene', {
    'operation': 'validate',
    'data': memory_entry
})

if validation['valid']:
    write_to_memory(memory_entry)
else:
    handle_validation_errors(validation['errors'])

Periodic Maintenance

# Daily cron job
0 2 * * * node ~/.claude/memory/skill-executor.js execute memory-hygiene '{"operation":"expire"}'

# Weekly compaction
0 3 * * 0 node ~/.claude/memory/skill-executor.js execute memory-hygiene '{"operation":"compact"}'

After Bulk Operations

# After importing many entries
import_entries(bulk_data)

# Immediately compact and deduplicate
execute_skill('memory-hygiene', {
    'operation': 'compact_and_merge'
})

Safety Measures

All hygiene operations:

  1. Create backup before modification (/memories/.backups/)
  2. Log all changes to ~/.claude/memory/hygiene.log
  3. Dry-run mode available with --preview flag
  4. Atomic operations - rollback on failure
  5. Preserve provenance - track all transformations

Examples

Good Memory Entries

See references/examples.md for examples of well-formed entries.

Bad Memory Entries

See references/examples.md for anti-patterns to avoid.

Performance

  • Validation: <10ms per entry
  • Compaction: ~100ms per 1000 entries
  • Deduplication: ~200ms per 1000 entries
  • Expiration: ~50ms per 1000 entries

Error Handling

  • Invalid schema: Returns detailed error with field name
  • Corrupted data: Archives problematic entries separately
  • Disk full: Fails gracefully with clear error
  • Backup failure: Aborts operation, preserves original

Configuration

Edit assets/hygiene_config.json to customize:

  • Similarity threshold for merging (default: 0.8)
  • Archive location (default: /memories/.archive/)
  • Backup retention (default: 7 days)
  • Max memory size before auto-compact (default: 5MB)

Integration Points

Memory Tool Operations

  • insert → validate_schema()
  • replace → validate_schema()
  • delete → log_deletion()
  • rename → update_metadata()

Context Editing

After context clearing, run compaction to optimize memory.

Background Tasks

Set up cron jobs for automated maintenance (see Integration section).


Memory Hygiene Skill v1.0.0 - Keeping your memory clean and queryable