emergency-ulimit-mitigation

Emergency steps to mitigate ulimit-related crashes by increasing file descriptor limits and restarting services.

⭐4stars🔀0forks📁View on GitHub🕐Updated Dec 29, 2025⬇️1downloads

When & Why to Use This Skill

This Claude skill provides a critical emergency runbook for mitigating system crashes caused by ulimit restrictions and file descriptor exhaustion. It offers immediate commands to increase system limits, monitor resource usage in real-time, and restart essential services like Docker and Traefik, ensuring rapid recovery and improved system stability during high-load incidents.

Use Cases

  • Scenario 1: Rapidly resolving 'Too many open files' errors on production Linux servers to prevent or recover from service downtime.
  • Scenario 2: Emergency recovery of containerized environments and reverse proxies (e.g., Docker, Traefik) after resource exhaustion leads to service failure.
  • Scenario 3: Real-time monitoring and auditing of file descriptor usage to identify leaking processes or capacity bottlenecks before they cause a crash.
  • Scenario 4: Deploying automated cleanup scripts and persistent ulimit configurations to harden system infrastructure against future resource-related instability.
nameemergency-ulimit-mitigation
descriptionEmergency steps to mitigate ulimit-related crashes by increasing file descriptor limits and restarting services.

qqu='qq "cat ~/d/AI/Prompts/Commands/fix-ulimit-exceeded.md"'# Emergency ulimit Mitigation

Tags: AI/Prompt, ulimit, filedescriptors, crashing, toomanyfilesopen, Stability, Maintenance
Modified: 2025-10-27T08:55:02-04:00

Immediate Emergency Fix

# Instant relief - run immediately when services are crashing
ulimit -n 65536
echo 2097152 | sudo tee /proc/sys/fs/file-max

Emergency Service Recovery

# Restart crashed services
sudo systemctl restart docker traefik
sudo docker system prune -f

Check Current Status

# See what's using FDs
lsof | wc -l
cat /proc/sys/fs/file-nr | awk '{printf "FDs: %d/%d (%.1f%%)\n", $1, $3, ($1/$3)*100}'

Deploy Full Solution

sudo ./scripts/set-ulimits.sh
sudo systemctl enable --now fd-monitor.timer

Emergency Cleanup

./scripts/fd-cleanup.sh