Query performance is slow with larger amount of items

[skoshx]

With about 7000 items, an API GET request with a findMany() query takes about 8 seconds. The duration of the query (linearly?) correlates with the amount of items.

[jnssnmrcs]

Might be a good idea to add some benchmarks that run in CI with deno bench for different amount of items and scenarios.

[skoshx]

Yeah, that was definitely the intention, just haven't gotten around to doing so... And it wasn't a super high priority as I didn't notice any performance problems, but now my OG Image generation is taking 8 seconds and that is untenable so will have to prioritize this haha :D

[skoshx]

Quite fruitless efforts, running benchmarks on my local machine atleast, with 10,000 items, I'm getting p995 times of 24 microseconds on findFirst & findMany… Will need to do some different kind of benchmarking to understand why it's being so slow in production.

[skoshx]

For posterity; benchmark was written late at night and as such it was incorrect, here are the real results:

🍔 Created 10000 mock double cheeseburger orders. cpu: Apple M2 Pro runtime: deno 1.34.1 (aarch64-apple-darwin)

file:///Users/rasmus/Desktop/DEV/Web/pentagon/bench/crud_bench.ts benchmark time (avg) (min … max) p75 p99 p995

---

findFirst > where 40.17 ms/iter(37.89 ms … 42.32 ms) 41.57 ms 42.32 ms 42.32 ms findMany 42.07 ms/iter(39.83 ms … 45.69 ms) 43.25 ms 45.69 ms 45.69 ms findMany > where 43.29 ms/iter(41.57 ms … 45.03 ms) 44.29 ms 45.03 ms 45.03 ms findMany > where & include 99.47 ms/iter(96.27 ms … 103.38 ms) 100.88 ms 103.38 ms 103.38 ms

include causes major performance hit.

[skoshx]

The latest 0.2.1 release no longer has performance problems (aside from include being about 2x slower), so closing for now.

[waptik]

The latest 0.2.1 release no longer has performance problems (aside from include being about 2x slower), so closing for now.

Version 0.2.1 does not exist yet on deno x

[skoshx]

With a large amount of items, there's performance problems on the edge. Will have to investigate whether it's because of Pentagon's bad implementation or something on Deno KV's end.