itertools
itertools copied to clipboard
Mutable "stripes" of data
I asked about this in Rayon, but the functionality is not actually specific to parallelism, so I've copied my request here instead:
Is there a way to get "chunks"-like functionality but with striping/stepping instead of contiguous memory?
I.e., if you have some data in a slice, and every nth element of this slice is part of the same independent workload, you could split up the buffer for processing on multiple threads in such a way that the data for each thread would interleaved with the other threads' data.
This doesn't seem to be the semantics of
interleave,windows,step_by, or any of the other functions I've found so far in the documentation.I am told that such an access pattern is common in image processing.
ndarray implements a relatively efficient way to do that — array views, and the striping is actually the same thing as arranging the data into rows and columns (either of those being the stripe).
If you say image processing, then I'm thinking of high performance and the general iterator trait might not be the best for that.
Tying back to rayon, there's also the ndarray-parallel crate...
(and isn't ndarray itself supposed to be getting rayon support?)
Yes! I'm sorry, it hasn't had a release for a few months. 0.13 has rayon support directly.
@bluss Yes, that sounds very much like what I was thinking of! I will check how close ndarray-parallel is to what I'm picturing.