arrow2 icon indicating copy to clipboard operation
arrow2 copied to clipboard

Arrow2 memory usage too much in `lexsort_to_indices` than arrow

Open sundy-li opened this issue 2 years ago • 1 comments

Which is found by : https://github.com/datafuselabs/datafuse/issues/1274

Seems we forget to add shrink_to_fit in

https://github.com/jorgecarleitao/arrow2/blob/fa3c2ce2dd4419256de8283f885538b6b011e5d4/src/compute/sort/lex_sort.rs#L175-L179

and some other places.

sundy-li avatar Aug 04 '21 06:08 sundy-li

Seems not related to shrink_to_fit , because it already has shrink_to_fit in indices_sorted_unstable_by.

need to profile the memory to see the differ :)

sundy-li avatar Aug 04 '21 07:08 sundy-li