feat: add queue size warning logs for high parallelism detection

[jchris]

Summary

Implements warning logs to detect when applications hit the database with excessive parallelism, helping identify performance bottlenecks before they become critical issues.

Changes

ApplyHeadQueue Monitoring

• Warning logs when applyHeadQueue size exceeds 5 items
• Debug logs for all operations showing queue size and local update context
• Added in

WriteQueue Monitoring

• Warning logs when writeQueue size exceeds 10 items
• Debug logs for all bulk operations showing queue size and task count
• Added in

Comprehensive Test Suite

• Queue behavior tests - Normal operations, high concurrency scenarios
• ApplyHeadQueue specific tests - Task sorting, error handling, size tracking
• WriteQueue specific tests - Chunking, concurrent operations, closure handling
• Tests validate logging doesn't impact functionality or performance

Benefits

• Early Warning System: Detect queue buildup before performance degrades
• Performance Tuning: Help developers optimize high-concurrency access patterns
• Debugging Aid: Correlate performance issues with queue sizes during stress testing
• Operational Visibility: Monitor queue health in production environments

Testing

The feature can be tested by creating high-concurrency scenarios:

// Trigger applyHeadQueue warnings (>5 items)
const promises = Array.from({ length: 20 }, (_, i) => 
  db.put({ id: i, data: `concurrent-${i}` })
);

// Trigger writeQueue warnings (>10 items)  
const bulkPromises = Array.from({ length: 15 }, (_, i) => {
  const docs = Array.from({ length: 5 }, (_, j) => ({ batch: i, item: j }));
  return db.bulk(docs);
});

await Promise.all([...promises, ...bulkPromises]);

Resolves

Closes #1053

Future Enhancements

• Configurable warning thresholds
• Metrics/telemetry integration
• Adaptive backpressure mechanisms

Summary by CodeRabbit

• New Features
  • Enhanced observability: added debug logs for queue sizes and bulk operations; warnings when thresholds are exceeded to highlight potential high concurrency.
• Tests
  • Introduced comprehensive suites validating logging behavior, write-queue operations (push/bulk), chunking, concurrency, ordering, error propagation, and database operations under load.
• Chores
  • Instrumentation-only update with no functional or API changes.

[coderabbitai[bot]]

Walkthrough

Adds debug and warning logs to CRDT apply-head and write queue to report queue sizes (warn thresholds: >5 and >10). Introduces runtime tests exercising logging paths, concurrency, ordering, and error propagation with mocked workers/DB. No public API changes or algorithm modifications.

Changes

Cohort / File(s)	Summary
Runtime logging: CRDT clock core/base/crdt-clock.ts	Adds queue size measurement in int_applyHead; emits debug logs always and warning logs when size > 5; includes localUpdates context. No control-flow or API changes.
Runtime logging: Write queue core/base/write-queue.ts	In WriteQueueImpl.bulk, logs writeQueue size and bulk task count (debug always, warn when size > 10). No changes to processing logic or signatures.
Tests: ApplyHeadQueue logging core/tests/runtime/apply-head-queue-logging.test.ts	New Vitest suite validating applyHeadQueue logging-enabled behavior with mocked worker/logger; covers queue size, localUpdates flag, concurrency, ordering, and error propagation.
Tests: DB runtime logging scenarios core/tests/runtime/queue-logging.test.ts	New Vitest suite driving high-parallelism DB operations (put/bulk/allDocs/get) with mocked logger; validates completion and result structures under concurrency.
Tests: WriteQueue logging and behavior core/tests/runtime/write-queue-logging.test.ts	New Vitest suite for writeQueue with mocked worker/logger; covers bulk/push, chunking, concurrency, close(), errors, mixed ops, and ordering.

Sequence Diagram(s)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~18 minutes

Assessment against linked issues

Objective	Addressed	Explanation
ApplyHeadQueue: debug logs show queue size on every int_applyHead (#1053)	✅
ApplyHeadQueue: warning when queue size exceeds 5; include localUpdates context (#1053)	✅
WriteQueue: debug logs show queue size and bulk task count on every operation (#1053)	✅
WriteQueue: warning when queue size exceeds 10 (#1053)	✅

Possibly related PRs

• fireproof-storage/fireproof#467 — Also modifies write-queue behavior around bulk handling; likely related to observability or batching semantics.

Suggested reviewers

• mabels

✨ Finishing Touches

• [ ] 📝 Generate Docstrings

🧪 Generate unit tests

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment
• [ ] Commit unit tests in branch jchris/queue-warn

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[mabels]

I will pull a queue implementation (did a lot before) into @adviser/cement that has this feature built into it.

To fix my ResolveOnce reset problem, I need that anyway.