vortex icon indicating copy to clipboard operation
vortex copied to clipboard

Investigate why `VortexSession` change introduced vortex:duckdb perf regression for clickbench

Open 0ax1 opened this issue 1 month ago • 5 comments

https://github.com/vortex-data/vortex/commit/879a53baab7b2a4f72e928fba3cbf0fb92eb989a caused some Clickbench queries to regress for vortex:duckdb.

0ax1 avatar Nov 04 '25 11:11 0ax1

@gatesn Do you have a hunch?

0ax1 avatar Nov 04 '25 11:11 0ax1

Seems like #5198 was majority of it

robert3005 avatar Nov 05 '25 17:11 robert3005

There's still 10% regression in some queries

robert3005 avatar Nov 05 '25 17:11 robert3005

So both session and Registry now hold dashmaps. I wonder if that's wrong, and we should actually hold HashMap? Or perhaps the deserialization logic should take an immutable snapshot of the registry prior to deserializing?

gatesn avatar Nov 05 '25 19:11 gatesn

When I was looking at this before the locking cost was non negligible and ideally we’d take immutable snapshot and have a read only version

robert3005 avatar Nov 05 '25 20:11 robert3005