spring-batch icon indicating copy to clipboard operation
spring-batch copied to clipboard

Spring Batch is very slow when using two datasources (one for batch/task and another for the app)

Open mlouis08 opened this issue 3 years ago • 0 comments
trafficstars

Bug description Spring batch configured with two datasources (one for batch/task and another for the app) is much slower than using one datasource for both (boot configured or custom). Following the spring recommended approach for two datasources (here and here) does not help. Tried this on the sample file-to-jdbc example provided by spring. The datasources (in my case) point to the same MySql DB server but to different schemas.

Timing observed with the provided input file "data/name-list.csv" in the example:

  1. Default Boot configured datasource - 1 second
  2. One custom datasource (for both Batch/Task and App) - 1 second
  3. Two custom datasources (one each for Batch/Task and App) - 90 seconds

Environment Java 17, Spring Boot 2.4.x, MySql 8 running on Ubuntu 20.04

Steps to reproduce Run the file-to-jdbc sample app provided by spring with two data sources created per the recommendations in links above in bug description. More details below on the Minimal complete reproducible section.

Expected behavior Expect to see the similar timing as one datasource for both batch/task and app

Minimal Complete Reproducible example I used the sample file-to-jdbc as the baseline and changed it to use two datasources. Modified files attached in zip. file-to-jdbc-ChangedFiles.zip

mlouis08 avatar Feb 19 '22 20:02 mlouis08