Allow usage of separate DB connection pools for request handling and background processing
Current Behavior
Dependency-Track maintains a single database connection pool. The pool defaults to a maximum of 20 connections.
The pool is shared by request handling, and background processing logic. This could cause issues when an increased volume of background tasks coincides with many concurrent users interacting with the REST API.
Proposed Behavior
To prevent API responsiveness from being compromised by background task processing, and vice versa, offer the option to maintain separate connection pools for each.
This would only be required for deployments of certain sizes and should thus not be the default.
Checklist
- [x] I have read and understand the contributing guidelines
- [x] I have checked the existing issues for whether this enhancement was already requested
This could proof painful to implement with DataNucleus:
- Data sources are configured at the
PersistenceManagerFactorylevel, so we'd need two separate PMFs. - Alpine is currently based on the idea that there'll only ever be one PMF: https://github.com/stevespringett/Alpine/blob/master/alpine-server/src/main/java/alpine/server/persistence/PersistenceManagerFactory.java
DataNucleus already maintains two separate "connection factories": One for transactional, and one for non-transactional operations.
Assuming write-operations are always wrapped in a transaction, this provides a separation between read/write and read-only operations.
For JDBI, we currently always use the "primary" connection factory, which is what DN would use for transactional operations. Perhaps it is possible to use the "secondary" connection factory for operations that we explicitly label as read-only.
We would further need to ensure that write operations performed via DataNucleus indeed consistently use transactions.