image icon indicating copy to clipboard operation
image copied to clipboard

Chunks type function for potential data parallelism

Open arthmis opened this issue 5 years ago • 6 comments

Is it possible to get an iterator like the chunks iterator because I'm trying to write code to parallel manipulate the image using crossbeam::scope threads? However, I can't convince the compiler that I'm operating over non overlapping regions of the image when I split it up into sub images. Chunks used in Vec

arthmis avatar May 04 '19 13:05 arthmis

If the implementor of GenericImage is allowed to choose the chunk size, then maybe it is possible to work around the language level issues discussed in #888. For some images, the intended solution for ffi FlatSamples could help you out. Create it by ImageBuffer::as_flat_samples etc.

HeroicKatora avatar May 04 '19 14:05 HeroicKatora

Thank you. Would it be possible to have a chunks like iterator for the flat samples?

Also what would I have to look into to let GenericImage choose the chunk size? I'm new to this so I don't know where to start.

arthmis avatar May 04 '19 14:05 arthmis

Would implementing Parallel iterator from rayon be an option?

arthmis avatar May 07 '19 17:05 arthmis

Possibly, but we can not introduce a hard dependency on rayon due to the wasm target as far as I am aware. So only behind a non-default feature flag or some other switch.

HeroicKatora avatar May 07 '19 17:05 HeroicKatora

As far as I understand when https://github.com/rayon-rs/rayon/pull/636 lands, wasm support in rayon should be right around the corner. Until then having some sort of rayon support behind a feature gate would be nice.

birktj avatar May 07 '19 17:05 birktj

Possibly, but we can not introduce a hard dependency on rayon due to the wasm target as far as I am aware. So only behind a non-default feature flag or some other switch.

Note that parallelisation in Wasm is supported nowadays and we recently published an adapter for Rayon + Wasm: https://github.com/RReverser/wasm-bindgen-rayon

Incidentally, even though I wrote that adapter for work & OSS, I now need it in side project where I'm dealing with images and was hoping to parallelise some image operations as well - that's how I found this issue :)

RReverser avatar May 25 '21 01:05 RReverser