dub icon indicating copy to clipboard operation
dub copied to clipboard

Add retry mechanism for failed requests

Open devkiran opened this issue 2 months ago β€’ 2 comments

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Improved error handling for cron jobs and API endpoints with standardized response formats.
    • Added automatic retry capability for transient database errors on failed API requests.
    • Enhanced error response headers to properly indicate retryable failures to external systems.
  • Style

    • Minor formatting adjustments to UI components and CSS class ordering.

✏️ Tip: You can customize this high-level summary in your review settings.

devkiran avatar Dec 10 '25 16:12 devkiran

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
dub Ready Ready Preview Jan 8, 2026 6:32am

vercel[bot] avatar Dec 10 '25 16:12 vercel[bot]

πŸ“ Walkthrough

Walkthrough

Replaces positional error handler usage across many cron routes with a new cron-specific wrapper, refactors the central error handler to accept an object and detect QStash callbacks / transient Prisma errors, adds queueFailedRequestForRetry (QStash retry publisher), and updates non-cron routes/auth call sites to the new error-handler parameter shape.

Changes

Cohort / File(s) Change Summary
Cron routes β€” error handler migration
apps/web/app/(ee)/api/cron/... (many files, e.g. aggregate-clicks/route.ts, bounties/*, commissions/export/route.ts, imports/*, links/*, payouts/*, streams/*, workflows/*, etc.)
Replaced imports/usages of handleAndReturnErrorResponse with handleCronErrorResponse (from local ../utils) and changed catch blocks to call handleCronErrorResponse({ error }). No other control-flow changes in routes.
New cron error helper
apps/web/app/(ee)/api/cron/utils.ts
Added exported handleCronErrorResponse({ error }) that maps errors via handleApiError and returns NextResponse.json({ error: apiError }) with the mapped status; logAndRespond retained.
Core error handler refactor
apps/web/lib/api/errors.ts
Refactored handleAndReturnErrorResponse to accept a single options object ({ error, responseHeaders?, requestHeaders? }), exported handleApiError, added Prisma import and detection of transient DB errors, detects QStash callbacks (via upstash-signature), and sets retry/non-retry headers/status (including 489) for callback cases.
Retry queue utility (QStash)
apps/web/lib/api/queue-failed-request-for-retry.ts
New queueFailedRequestForRetry({ error, apiKey, req }) that enqueues retry jobs to QStash for transient Prisma errors with jitter/exponential backoff, guards against callback loops, logs results, and limits invocation per request.
Non-cron routes & webhook updates
apps/web/app/... (e.g. api/hubspot/*, api/singular/webhook/route.ts, api/shopify/pixel/route.ts, api/track/*, api/qr/route.tsx, api/oauth/*, etc.)
Updated catch blocks to pass { error, responseHeaders? } (object form) to handleAndReturnErrorResponse (formerly positional arguments). Some routes renamed catch variable to error.
Auth / middleware call-sites
apps/web/lib/auth/*.ts, apps/web/lib/embed/referrals/auth.ts, apps/web/lib/integrations/shopify/process-order.ts
Updated calls to handleAndReturnErrorResponse to use the new options-object parameter. workspace.ts additionally imports queueFailedRequestForRetry and enqueues retryable failed requests in catch blocks; also minor conversion-logging path checks adjusted.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant API as App Route
    participant DB as Prisma
    participant Error as handleAndReturnErrorResponse
    participant Queue as QStash
    participant Logger

    Client->>API: Send request
    API->>DB: Perform DB operation
    DB-->>API: Throws transient DB error
    API->>Error: catch -> handleAndReturnErrorResponse({ error, requestHeaders, responseHeaders })
    Error->>Error: map error via handleApiError()
    Error->>Error: is QStash callback? (check upstash-signature)
    alt QStash callback & transient DB error
        Error->>Queue: publish retry job (jitter + backoff, 5 retries)
        Error->>Logger: log queued retry
        Error-->>API: return JSON error with retry headers/status
    else Standard request or non-retryable
        Error-->>API: return mapped JSON error & status
    end
    API-->>Client: Error response

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • dubinc/dub#3069 β€” Overlaps changes in apps/web/lib/api/errors.ts (signature/logic updates to core error handling).
  • dubinc/dub#3003 β€” Modifies apps/web/app/(ee)/api/cron/payouts/aggregate-due-commissions/route.ts, which this PR also touches for error-handler migration.
  • dubinc/dub#3121 β€” Touches apps/web/app/(ee)/api/cron/payouts/process/route.ts, overlapping cron error/logging adjustments.

Suggested reviewers

  • steven-tey

Poem

🐰
I hopped through code both dusk and dawn,
unified errors, nicely drawn.
QStash holds retries safe and spry,
transient faults we now defy.
A carrot-cheer for logs gone on high! ✨

πŸš₯ Pre-merge checks | βœ… 3
βœ… Passed checks (3 passed)
Check name Status Explanation
Description Check βœ… Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check βœ… Passed The title accurately reflects the main objective of the PR, which introduces a new retry mechanism for failed requests across multiple API endpoints.
Docstring Coverage βœ… Passed Docstring coverage is 80.88% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • [ ] πŸ“ Generate docstrings

πŸ“œ Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between c0eeb32ac366df3e01dbea81b4ad556629c753dc and e96ea920b78f0f51dc0a70240e659a425a9d9fe4.

πŸ“’ Files selected for processing (1)
  • apps/web/app/(ee)/api/cron/payouts/payout-failed/route.ts
🧰 Additional context used
🧠 Learnings (3)
πŸ““ Common learnings
Learnt from: devkiran
Repo: dubinc/dub PR: 3213
File: apps/web/app/(ee)/api/cron/auto-approve-partner/route.ts:122-122
Timestamp: 2025-12-15T16:46:01.529Z
Learning: In the Dub codebase, cron endpoints under apps/web/app/(ee)/api/cron/ use handleCronErrorResponse for error handling, which intentionally does NOT detect QStash callbacks or set Upstash-NonRetryable-Error headers. This allows QStash to retry all cron job errors using its native retry mechanism. The selective retry logic (queueFailedRequestForRetry) is only used for specific user-facing API endpoints like /api/track/lead, /api/track/sale, and /api/links to retry only transient Prisma database errors.
Learnt from: devkiran
Repo: dubinc/dub PR: 3207
File: apps/web/lib/cron/with-cron.ts:27-56
Timestamp: 2025-12-09T12:54:41.818Z
Learning: In `apps/web/lib/cron/with-cron.ts`, the `withCron` wrapper extracts the request body once and provides it to handlers via the `rawBody` parameter. Handlers should use this `rawBody` string parameter (e.g., `JSON.parse(rawBody)`) rather than reading from the Request object via `req.json()` or `req.text()`.
πŸ“š Learning: 2025-12-15T16:45:51.667Z
Learnt from: devkiran
Repo: dubinc/dub PR: 3213
File: apps/web/app/(ee)/api/cron/auto-approve-partner/route.ts:122-122
Timestamp: 2025-12-15T16:45:51.667Z
Learning: In cron endpoints under apps/web/app/(ee)/api/cron, continue using handleCronErrorResponse for error handling. Do not detect QStash callbacks or set Upstash-NonRetryable-Error headers in these cron routes, so QStash can retry cron errors via its native retry mechanism. The existing queueFailedRequestForRetry logic should remain limited to specific user-facing API endpoints (e.g., /api/track/lead, /api/track/sale, /api/links) to retry only transient Prisma/database errors. This pattern should apply to all cron endpoints under the cron directory in this codebase.

Applied to files:

  • apps/web/app/(ee)/api/cron/payouts/payout-failed/route.ts
πŸ“š Learning: 2025-12-09T12:54:41.818Z
Learnt from: devkiran
Repo: dubinc/dub PR: 3207
File: apps/web/lib/cron/with-cron.ts:27-56
Timestamp: 2025-12-09T12:54:41.818Z
Learning: In `apps/web/lib/cron/with-cron.ts`, the `withCron` wrapper extracts the request body once and provides it to handlers via the `rawBody` parameter. Handlers should use this `rawBody` string parameter (e.g., `JSON.parse(rawBody)`) rather than reading from the Request object via `req.json()` or `req.text()`.

Applied to files:

  • apps/web/app/(ee)/api/cron/payouts/payout-failed/route.ts
🧬 Code graph analysis (1)
apps/web/app/(ee)/api/cron/payouts/payout-failed/route.ts (1)
apps/web/app/(ee)/api/cron/utils.ts (1)
  • handleCronErrorResponse (22-33)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build
πŸ”‡ Additional comments (1)
apps/web/app/(ee)/api/cron/payouts/payout-failed/route.ts (1)

8-8: LGTM! Error handling refactoring is correct.

The changes successfully migrate this cron endpoint to use the new handleCronErrorResponse wrapper with object parameter syntax. The implementation aligns with the established pattern for cron error handling in this codebase.

Based on learnings, this approach correctly allows QStash to handle retries using its native retry mechanism for all cron job errors.

Also applies to: 88-88


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Dec 10 '25 16:12 coderabbitai[bot]