feat(core): add redis lock during insertion of event in event table during initial attempt of outgoing webhook delivery

[dgeee13]

Type of Change

• [ ] Bugfix
• [ ] New feature
• [X] Enhancement
• [ ] Refactoring
• [ ] Dependency updates
• [ ] Documentation
• [ ] CI/CD

Description

The current implementation of insertion of an event in event table does not handle the case where multiple requests with the same idempotent_event_id might be processed concurrently, leading to multiple unique constraint violation errors in the database.

This was reported by a merchant who was facing high volumes of unique constraint violation errors.

NOTE: this unique constraint log error issue is only applicable for theinitial attempt of outgoing webhook delivery, not for retry attempts

Why does this happen?

1. Multiple requests with the same idempotent_event_id might reach the database at the same time.
2. Each request checks for the event, finds nothing (since it’s not committed yet), and proceeds to insert.
3. The first request succeeds, while the others fail due to the unique constraint on idempotent_event_id.

Solution (Using Redis Lock) Before querying the database, acquire a lock in Redis for the given idempotent_event_id. If there are multiple processes with same idempotent_event_id only one process will hold the lock, others will leave early.

If a lock is acquired:

• Check if the event already exists in the database.
• If it exists, return early
• Otherwise, insert the event.
• Once done, release the lock

Additional Changes

• [ ] This PR modifies the API contract
• [ ] This PR modifies the database schema
• [ ] This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Checklist

• [x] I formatted the code cargo +nightly fmt --all
• [x] I addressed lints thrown by cargo clippy
• [x] I reviewed the submitted code
• [ ] I added unit tests for my changes where possible

[semanticdiff-com[bot]]

Review changes with  

Changed Files

File	Status
crates/router/src/core/webhooks/outgoing.rs	17% smaller
config/config.example.toml	Unsupported file format
config/deployments/integration_test.toml	Unsupported file format
config/deployments/production.toml	Unsupported file format
config/deployments/sandbox.toml	Unsupported file format
config/development.toml	Unsupported file format
config/docker_compose.toml	Unsupported file format
crates/router/src/configs/settings.rs	0% smaller
crates/router/src/configs/validations.rs	0% smaller
crates/router/src/core/webhooks/utils.rs	0% smaller
loadtest/config/development.toml	Unsupported file format