Transducers.jl icon indicating copy to clipboard operation
Transducers.jl copied to clipboard

Improve performance of `collect` on size unstable collections.

Open MasonProtter opened this issue 2 years ago • 0 comments

Okay, so the low hanging fruit of SizeStable collections should be dealt with in https://github.com/JuliaFolds/Transducers.jl/pull/553, but SizeChanging collections also need attention:

julia> let v = randn(10000)
           @btime filter(x -> x < 1000, $v)
           @btime collect(Filter(x -> x < 1000), $v)
       end
  4.767 μs (2 allocations: 78.17 KiB)
  7.820 μs (10 allocations: 326.61 KiB)

Perhaps the most catastrophic offender I've seen is Partiton.

julia> let v = rand(10000), sz = 6
           @btime collect(Iterators.partition($v, $sz))
           @btime collect(Partition($sz), $v)
       end;
  3.197 μs (2 allocations: 65.17 KiB)
  18.825 ms (189965 allocations: 6.52 MiB)

MasonProtter avatar May 05 '23 00:05 MasonProtter