redisql icon indicating copy to clipboard operation
redisql copied to clipboard

[BUG] CopyToString is over 100% slower than other methods

Open DGKSK8LIFE opened this issue 3 years ago • 2 comments

Expected Behavior

Performance in CopyToString should be comparable to that of CopyToList and CopyToHash.

Current Behavior

CopyToString is over 100% slower than the aforementioned two.

Possible Solution

Because of Redis' data structures, CopyToString (strings) has to write one key per field, compared to CopyToList and CopyToHash that both write one key per row. This is not necessarily something that can be "quick fixed", but with other performance optimizations, especially transactions and pipelining, performance should improve drastically.

Steps to Reproduce

  1. Run test cases with ~10k rows or so and your preferred DB.
  2. View output:
➜  redisql git:(master) go test -v -db mysql -rows 10000 
Preparing Test...
=== RUN   TestCopyToString
--- PASS: TestCopyToString (8.43s)
=== RUN   TestCopyToList
--- PASS: TestCopyToList (0.75s)
=== RUN   TestCopyToHash
--- PASS: TestCopyToHash (0.92s)
PASS
ok      github.com/DGKSK8LIFE/redisql   17.995s

DGKSK8LIFE avatar Sep 29 '21 02:09 DGKSK8LIFE

Hi @DGKSK8LIFE , have you solved this yet? I'd like to try to solve this

Fajrinmk avatar Sep 30 '21 10:09 Fajrinmk

I'm working on one possible solution, but the bug is not definitively resolved. Definitely want to see your solution so go ahead and work on this, ty. @Fajrinmk

DGKSK8LIFE avatar Sep 30 '21 12:09 DGKSK8LIFE