dice icon indicating copy to clipboard operation
dice copied to clipboard

Make raw string as `[]byte` instead of `string` for efficiency

Open arpitbbhayani opened this issue 3 years ago • 7 comments

Converting string to []byte is not a typecast but a mem copy and rebuilding, hence it becomes expensive.

arpitbbhayani avatar Sep 28 '22 10:09 arpitbbhayani

@arpitbbhayani I didn't understood your question. In title you have written that you want this change for efficiency, but in above comment you have written that it is expensive. If mem copy and rebuilding operation is expensive, then it will decrease the efficiency right?

deeplearningera avatar Oct 18 '22 15:10 deeplearningera

@arpitbbhayani where do we have to make changes ?

heysujal avatar Oct 20 '22 10:10 heysujal

@deeplearningera "Converting string to []byte is not a typecast" implies while processing when we do this, it will be expensive. Hence we should not have string at all and instead just []byte.

@heysujal would recommend you go through the code and find a place where we are using RAW encoding.

arpitbbhayani avatar Oct 20 '22 11:10 arpitbbhayani

https://github.com/DiceDB/dice/blob/c9ac91786788b709b65440e488545099861bc408/core/type_string.go#L7 found this. I am not familiar with Golang and also to redis. So, only changing v string to []byte(v) would do the job ?

https://github.com/DiceDB/dice/blob/c9ac91786788b709b65440e488545099861bc408/core/object.go#L16 this also uses raw encoding

heysujal avatar Oct 20 '22 12:10 heysujal

Came across an interesting blog on this topic: https://syslog.ravelin.com/byte-vs-string-in-go-d645b67ca7ff Anyone working on this can refer this to get an insight as to why its a memory allocation and not just simple type-casting

asutosh97 avatar Oct 30 '22 18:10 asutosh97

@asutosh97 Still a valid issue? cc - @JyotinderSingh @arpitbbhayani

AshwinKul28 avatar Aug 16 '24 06:08 AshwinKul28

@AshwinKul28 / @JyotinderSingh anyone looking to work on this ?

sbshah97 avatar Aug 28 '24 23:08 sbshah97

Thanks @sbshah97 for the interest. Unfortunately, this was a very old issue and we are revamping the store, and query operations. This issue is not relevant now; though we will open more relevant ones soon after the rewrite. Closing this.

AshwinKul28 avatar Sep 08 '24 11:09 AshwinKul28