soda-for-java icon indicating copy to clipboard operation
soda-for-java copied to clipboard

Clients cannot specify batch size for inserts

Open osawyerr opened this issue 4 years ago • 1 comments

In the SODA driver the batch size used in the PreparedStatement for persisting a collection of documents is hardcoded to 100. It would be a useful addition to enable the caller to specify the batch size themselves to override the default and optimise inserts for their workloads. For example, in our tests for our workload setting the batch size to 10,000 when using raw JDBC to persist 100,000 JSON (BLOB) documents results in a massive performance increase over using 100.

Perhaps it would be useful to let the caller specify what batch size to use to optimise their inserts. Something along the lines of:

public List<OracleDocument> insertAndGet(Iterator<OracleDocument> documents, int batchSize)
    throws OracleBatchException;

alternatively allowing callers to specify the batch size to use in the "hints" options.

public List<OracleDocument> insertAndGet(Iterator<OracleDocument> documents, Map<String, ?> options)
    throws OracleBatchException;

Thoughts?

osawyerr avatar Dec 11 '20 15:12 osawyerr

Thank you for the suggestion, makes sense. Will look into adding this.

morgiyan avatar Dec 11 '20 19:12 morgiyan