claude-code icon indicating copy to clipboard operation
claude-code copied to clipboard
Published 1 week ago • verified publisher icon anthropics

Excessive File I/O During Sessions (~1.5 writes/second to .claude.json)

Open pentafive opened this issue 1 week ago • 2 comments

Summary

Claude Code performs excessive file I/O operations during normal sessions, writing to .claude.json at an extremely high rate (~1.5 writes/second). A 20-minute session resulted in 1,860 atomic file writes and generated a 2.5 MB debug log with 28,307 lines.

Environment

  • Claude Code Version: 2.0.72
  • Platform: Linux (WSL2)
  • OS Version: Linux 6.6.87.2-microsoft-standard-WSL2
  • Shell: bash
  • Working Directory: ~/projects/MyProject/
  • Debug Log: ~/.claude/debug/399de4cf-7eca-4e6c-973f-f75dc86448e5.txt

Detailed Description

Observed Behavior

During a normal Claude Code session, the system continuously writes to ~/.claude.json (the main state file). Each write follows this pattern:

[DEBUG] Writing to temp file: ~/.claude.json.tmp.73914.1766019293909
[DEBUG] Preserving file permissions: 100644
[DEBUG] Temp file written successfully, size: 103774 bytes
[DEBUG] Applied original permissions to temp file
[DEBUG] Renaming ~/.claude.json.tmp.73914.1766019293909 to ~/.claude.json
[DEBUG] File written atomically

Session Statistics

Session Duration: 20 minutes 23 seconds (00:34:31 to 00:54:54 UTC) Total File Writes: 1,860 writes to .claude.json Write Rate: ~1.5 writes/second (~91 writes/minute) Debug Log Size: 2.5 MB (28,307 lines) Average File Size: ~103-104 KB per write

I/O Pattern

File writes occur:

  • After every user message
  • After every AI response chunk/stream event
  • During tool execution
  • During permission updates
  • During file history snapshots
  • During LSP diagnostic checks
  • During hook command checks

Example from debug log:

00:40:41 - Write to .claude.json (103,773 bytes)
[2m 21s gap with xclip error]
00:54:53 - Write to .claude.json (103,774 bytes)
00:54:53 - Write to .claude.json (103,774 bytes)  [0.035s later]
00:54:53 - Write to .claude.json (103,774 bytes)  [0.038s later]

Multiple writes within milliseconds of each other suggest redundant state persistence.

Impact Assessment

Performance Impact

Storage:

  • Constant SSD wear from 90+ writes/minute
  • Temp file creation/deletion churn
  • Debug log bloat (2.5 MB for 20 minutes)

System Resources:

  • File system cache pressure
  • Atomic write overhead (create temp → write → chmod → rename)
  • Debug logging overhead

User Experience:

  • Potential session lag/unresponsiveness during I/O spikes
  • May contribute to session hangs

Frequency: Always

  • Affects all Claude Code sessions
  • Rate appears consistent regardless of activity level
  • Even idle sessions may write frequently

Data Loss Risk: None

  • Frequent writes actually protect against data loss
  • Atomic write pattern is safe
  • Issue is performance, not correctness

Root Cause Analysis

Potential causes for excessive writes:

  1. State Persistence Strategy:

    • Claude Code may persist entire session state after every event
    • No write coalescing or debouncing
    • Every message/tool/event triggers immediate save

  2. File History Snapshots:

    [DEBUG] FileHistory: Making snapshot for message <UUID>
[DEBUG] FileHistory: Added snapshot for <UUID>, tracking N files

    Creates snapshots for every message, each triggering a write

  3. Hook Command Checks:

    [DEBUG] Hooks: Found 0 total hooks in registry
[DEBUG] Hooks: checkForNewResponses returning 0 responses

    Appears to run after every event, even when no hooks configured

  4. Redundant Writes: Multiple writes within milliseconds (same file size) suggest:

    • Race conditions between different subsystems
    • Lack of debouncing/throttling
    • Overlapping save triggers

Reproduction Steps

  1. Start any Claude Code session: claude
  2. Have a normal conversation (ask questions, request file operations, etc.)
  3. Monitor debug log growth: 
    watch -n 1 'ls -lh ~/.claude/debug/latest'
  4. Count file writes after session: 
    grep -o "Writing to temp file:.*" ~/.claude/debug/latest | wc -l

Expected Result:

  • File writes occur at reasonable intervals (e.g., after message batches, on manual save)
  • Debug log grows proportionally to conversation complexity
  • Write rate: ~0.1-0.5 writes/second (conservative estimate)

Actual Result:

  • File writes occur constantly at ~1.5 writes/second
  • Debug log grows to 2.5 MB in 20 minutes
  • Write pattern shows redundancy and lack of throttling

Evidence from Debug Log

Write Pattern Sample (Tail of Session)

2025-12-18T00:54:53.909Z [DEBUG] Writing to temp file: ~/.claude.json.tmp.73914.1766019293909
2025-12-18T00:54:53.914Z [DEBUG] File ~/.claude.json written atomically
2025-12-18T00:54:53.947Z [DEBUG] Writing to temp file: ~/.claude.json.tmp.73914.1766019293947
2025-12-18T00:54:53.952Z [DEBUG] File ~/.claude.json written atomically
2025-12-18T00:54:53.985Z [DEBUG] Writing to temp file: ~/.claude.json.tmp.73914.1766019293985
2025-12-18T00:54:53.990Z [DEBUG] File ~/.claude.json written atomically

Three writes within 81 milliseconds (00:54:53.909 to 00:54:53.990), all writing the same data size (103,774 bytes).

Hook Command Check Pattern

[DEBUG] Getting matching hook commands for UserPromptSubmit with query: undefined
[DEBUG] Found 0 hook matchers in settings
[DEBUG] Matched 0 unique hooks for query "no match query" (0 before deduplication)

This pattern repeats hundreds of times despite having zero hooks configured. Each check appears to trigger a file write cycle.

Performance Recommendations

Option 1: Write Debouncing (Recommended)

Implement write debouncing with:

  • Minimum interval between writes (e.g., 5 seconds)
  • Queue state changes in memory
  • Flush on explicit events (user message sent, tool completed)
  • Emergency flush on process exit

Option 2: Differential State Updates

Instead of writing full 104 KB state file every time:

  • Track delta changes
  • Append to journal file
  • Periodic compaction to main state file

Option 3: Conditional Writes

Only write state file when:

  • Conversation state changes (new message, tool result)
  • User explicitly saves or exits
  • Critical state changes (permissions, settings)
  • Periodic checkpoint (e.g., every 60 seconds)

Skip writes for:

  • Hook checks when no hooks configured
  • LSP diagnostics when no LSP servers running
  • File history snapshots (keep in memory, flush on demand)

Suggested Monitoring

Add metrics to track:

  • Write operations per session
  • Average time between writes
  • Write triggers (which subsystem caused write)
  • State file size over time

Make this visible to users:

claude --stats
# File writes: 1860 (1.5/sec avg)
# Session duration: 20m 23s
# State file size: 104 KB

Related Issues

See Also:

  • #10505 - "[BUG] Memory leak and runaway write loop in claude --continue causes 28GB memory usage"
    • Extreme version of this issue (infinite loop instead of just frequent writes)
    • Same underlying problem (no write debouncing/throttling)

Files for Reference

  1. Debug Log: ~/.claude/debug/399de4cf-7eca-4e6c-973f-f75dc86448e5.txt (2.5 MB, 28,307 lines)
  2. Session State: ~/.claude.json (~104 KB)

Session ID: 399de4cf-7eca-4e6c-973f-f75dc86448e5

pentafive avatar Dec 19 '25 08:12 pentafive