opencode icon indicating copy to clipboard operation
opencode copied to clipboard

fix(storage): handle empty JSON files gracefully

Open coleleavitt opened this issue 3 days ago • 5 comments

Fixes #7889 Fixes #7715

Summary

Fixed "Unexpected end of JSON input" and "Unterminated string" errors in storage and SDK client.

1. SDK Client (both v1 and v2 client.gen.ts)

  • Handle empty responses with proper JSON parsing check
  • Added try-catch with contextual error messages for malformed JSON

2. Storage Layer - Empty Files (storage.ts - commit 23848ed32)

  • Handle empty JSON files gracefully in read() and update() functions
  • Check for empty files before parsing, throw NotFoundError instead of JSON parse error

3. Storage Layer - Corrupted Files (storage.ts - commit 061652ed3)

  • NEW: Detect null bytes and control characters in JSON files
  • NEW: Comprehensive JSON parse error handling with descriptive messages
  • NEW: Make stats command resilient to corrupted files

Root Cause (Corrupted Files)

Found 3 production storage files containing 1518 null bytes each instead of valid JSON:

  • .trim() check only catches whitespace, not null bytes
  • JSON.parse() crashed with "Unterminated string" error
  • stats command failed because Promise.all() had no error handling

Changes

SDK Client

// Read response as text first, only parse JSON if non-empty
const text = await response.text()
if (!text) return undefined
data = JSON.parse(text)

Storage - Empty File Check

const content = await Bun.file(target).text()
if (!content.trim()) {
  throw new NotFoundError({ message: `Empty file: ${target}` })
}

Storage - Corruption Detection (NEW)

// Detect null bytes and control characters
const hasControlCharacters = /[\x00-\x08\x0B\x0C\x0E-\x1F]/.test(content)
if (hasControlCharacters) {
  throw new NotFoundError({ message: `Corrupted file detected: ${target}` })
}

// Wrap JSON.parse with error handling
try {
  const result = JSON.parse(content)
  return result as T
} catch (e) {
  const message = e instanceof Error ? e.message : String(e)
  throw new NotFoundError({ message: `Failed to parse JSON from ${target}: ${message}` })
}

Stats Command Resilience (NEW)

// Skip corrupted files gracefully instead of crashing
const projects = await Promise.all(
  projectKeys.map((key) => Storage.read<Project.Info>(key).catch(() => undefined))
)

Testing

  • ✅ Detected and handled 3 corrupted files in production storage
  • opencode-dev stats works (processed 4,773 sessions successfully)
  • ✅ All type checks pass
  • ✅ Empty file handling works as expected

Files Changed

  • packages/opencode/src/storage/storage.ts - Empty + corrupted file handling
  • packages/opencode/src/cli/cmd/stats.ts - Resilient error handling
  • SDK client files (auto-generated) - Empty response handling

coleleavitt avatar Jan 12 '26 00:01 coleleavitt