iqps-go icon indicating copy to clipboard operation
iqps-go copied to clipboard

Batch Inserts into Db when Uplaoding Papers

Open rajivharlalka opened this issue 1 year ago • 3 comments

Is your feature request related to a problem? Please describe. Currently when multiple papers are uploaded, metadata for each paper is inserted in individual insert statement. These inserts can be batched into a single one for performance bonus.

rajivharlalka avatar Aug 29 '24 18:08 rajivharlalka

@rajivharlalka or @harshkhandeparkar please update the state of this issue to be reflected on the kanban.

proffapt avatar Sep 26 '24 13:09 proffapt

@rajivharlalka db god :nam:

harshkhandeparkar avatar Sep 27 '24 06:09 harshkhandeparkar

Not a priority issue, but a good first one. Would like some fresher to take it up.

rajivharlalka avatar Sep 29 '24 11:09 rajivharlalka

Putting this on hold since the change would make error handling a lot more complicated. Since we send a per-paper error message, it would be harder to handle db rollbacks if only one of the files fails to write. Also, since all would be inserted into the db at once, an error would fail to upload all papers. The performance benefit for our use case is most probably not worth the extra code complication.

Ref: https://metakgp.slack.com/archives/C05KTAL5ARM/p1729064846865189

harshkhandeparkar avatar Oct 17 '24 13:10 harshkhandeparkar