node-postgres icon indicating copy to clipboard operation
node-postgres copied to clipboard

Implement Batch query package

Open iamkhush opened this issue 2 years ago • 14 comments

Attempts to fix #2257

iamkhush avatar Mar 29 '23 21:03 iamkhush

whoah this is really awesome! I'm excited to help get this over the finishline & release it.

brianc avatar Mar 30 '23 15:03 brianc

@charmander any thoughts on this? I think it looks p great!

brianc avatar Mar 30 '23 19:03 brianc

@charmander any thoughts on this? I think it looks p great!

If it’s going to be a separate package instead of an API change to pg’s default query type – why in this repo?

charmander avatar Apr 06 '23 00:04 charmander

@brianc @charmander please have a look at the PR and provide feedback

iamkhush avatar Apr 15 '23 22:04 iamkhush

I had to make changes in the types/pg to get the build running again Here is the PR waiting for the maintainers (@brianc, @charmander ) https://github.com/DefinitelyTyped/DefinitelyTyped/pull/65348

iamkhush avatar May 02 '23 22:05 iamkhush

waouh, so happy to see that ! thx for your work @iamkhush ! 🎉

abenhamdine avatar May 16 '23 16:05 abenhamdine

I think this important question remains unanswered

If it’s going to be a separate package instead of an API change to pg’s default query type – why in this repo

in other words : are maintainers of this repo willing to take responsability for this new package ?

abenhamdine avatar May 22 '23 19:05 abenhamdine

In other words : are maintainers of this repo willing to take responsability for this new package ?

@abenhamdine I have followed what @brianc advised me here. But ofcourse it will be good to have confirmation again

iamkhush avatar Jun 06 '23 06:06 iamkhush

Moving it to draft to add typescript definitions

iamkhush avatar Jun 06 '23 08:06 iamkhush

Is this still alive / any help needed to get this done? I would like to use this myself.

hillac avatar Jul 25 '24 09:07 hillac

Hey @hillac , happy to continue the work if maintainers would confirm on what was asked above

iamkhush avatar Aug 31 '24 14:08 iamkhush

Given this is a fully separate package, I can just publish and use this on its own without having to maintain my own fork of node postgres right? I might do that.

Edit: tried it out on some micro benchmarks. For updates I saw no real advantage compared to building a single big parametrized query. And for more than 50 rows, the big parametrized query is significantly faster. For inserts I'm seeing worse results compared to a big parametrized query.

hillac avatar Mar 13 '25 02:03 hillac

@hillac could you show the code where it didnt work for you ? Im curious since the bench.ts file showed triple the qps I had seen otherwise.

iamkhush avatar Mar 23 '25 21:03 iamkhush

https://github.com/hillac/pg-batch-bench Pulled the benchmark out into it's own repo. Example output in the readme.

I should change it to do the inserts and updates multiple times to get some percentiles and means, but you can pretty clearly see a trend that just building a large parameterized query is faster.

The batch is much nicer to use though. And might be a bit faster for single inserts, but that could just be a jit warmup thing. Once again, needs better statistics.

(Edit: Added some more iterations of inserting 1, 2 and 3 rows and batch is definitely slower, so likely just a jit warmup issue.)

I think your benchmark compared multiple round trip single inserts/ updates, which is slower. This benchmark does a single query with all the rows in it.

hillac avatar Mar 24 '25 15:03 hillac