large-file-handling
This skill should be used when the user asks about "reading large files", "file too big", "context limit", "chunking files", "offset and limit", "file causing crash", or when encountering files over 10MB. Provides strategies for safely reading large files without exhausting context.
When & Why to Use This Skill
The Large File Handling skill is an essential optimization tool designed to overcome LLM context window limitations. It provides Claude with sophisticated strategies to process files exceeding 10MB or 25,000 tokens by implementing intelligent chunking, offset-based reading, and targeted searching. By preventing system hangs and context exhaustion, this skill ensures stable and efficient performance when dealing with massive codebases, logs, and structured datasets.
Use Cases
- Analyzing massive server logs: Use grep and tail strategies to isolate specific error patterns without loading the entire multi-gigabyte file into context.
- Navigating large source files: Read specific code blocks using 'offset' and 'limit' parameters to stay within the 25,000-token per-read constraint.
- Processing structured data: Efficiently query large JSON, CSV, or SQLite databases using specialized CLI tools like jq or sqlite3 instead of direct file ingestion.
- Context optimization: Automatically identify and exclude binary, minified, or redundant files using .claudeignore best practices to preserve the 200,000-token total context window.
- Pre-read safety checks: Utilize the /file-check command to evaluate file sizes and types before attempting operations that could cause system crashes.
| name | Large File Handling |
|---|---|
| description | This skill should be used when the user asks about "reading large files", "file too big", "context limit", "chunking files", "offset and limit", "file causing crash", or when encountering files over 10MB. Provides strategies for safely reading large files without exhausting context. |
| version | 1.0.0 |
Large File Handling for Claude Code
Context Limits
Claude Code has these file reading constraints:
- 25,000 tokens maximum per file read (~100KB of text)
- 200,000 tokens total context window
- Files over 50MB may cause system hangs
Safe Reading Strategies
Use Offset and Limit Parameters
Read specific portions instead of entire files:
Read file with offset=0 limit=500 # First 500 lines
Read file with offset=1000 limit=500 # Lines 1000-1500
Search Before Reading
Find relevant sections first:
grep -n "pattern" large-file.log | head -20
Then read only around matching line numbers.
Blocked File Types
Never attempt to read directly:
- Databases: .db, .sqlite, .sqlite3
- Archives: .zip, .tar, .gz, .7z
- Binaries: .exe, .dll, .so, .bin
- Large media: videos, large images
Alternative Approaches by File Type
| File Type | Alternative |
|---|---|
| SQLite DB | sqlite3 file.db "SELECT * FROM table LIMIT 10" |
| Large log | tail -1000 file.log or grep "ERROR" file.log |
| JSON (large) | jq '.specific.path' file.json |
| CSV (large) | head -100 file.csv or csvtool |
| PDF (large) | pdftotext file.pdf - | head -1000 |
| Minified JS | Usually not needed - read source instead |
.claudeignore Best Practices
Add these patterns to prevent accidental reads:
# Binary and database
*.db
*.sqlite
*.exe
*.dll
# Large generated files
*.min.js
*.bundle.js
# Dependencies
node_modules/
vendor/
.venv/
# Build output
dist/
build/
.next/
# Logs
*.log
logs/
Commands Available
/file-check <path>- Check if a file is safe to read/file-chunk <path>- Get chunking strategy for large file/scan-large-files [dir]- Find problematic files in directory
When Encountering Large Files
- Check size first using
/file-check - For text files > 10MB: Use chunking with offset/limit
- For logs: Search for relevant patterns first
- For databases: Use CLI tools to query
- For generated files: Often not needed - read source instead