ndarray icon indicating copy to clipboard operation
ndarray copied to clipboard

iterators are confusing

Open benkay86 opened this issue 4 years ago • 1 comments
trafficstars

The current framework for iteration over arrays is confusing. Issue edited because it's even more confusing than at first glance!

Common scenarios for iteration include:

  • Iterating over the elements of an array
  • Iterating over subviews along an axis of an array
  • Iterating mutably
  • Indexed iteration
  • Iterating over multiple arrays in lockstep
  • Parallel iteration with Rayon

Users may expect some consistent idioms with regard to how these different iterators are accessed. The ArrayBase documentation is the starting point for many users. It is currently non-obvious (at least to me) that:

  • We can iterate over subviews along an axis in parallel. Because there is ArrayBase::par_iter() but not ArrayBase::par_axis_iter().
  • We can have indexed iterators over subview along an axis. Because there is ArrayBase::indexed_iter() but not ArrayBase::indexed_axis_iter().

I may be missing it, but it appears there is no currently no way to:

  • Get a parallel version of IndexedIter without the overhead of Zip::indexed().
  • Get a (non-parallel) Iterator from Zip.

There are good reasons for some of these idiosyncrasies (e.g. expeditious blanket impls), but they still steepen ndarray's learning curve. Ways to address this would include:

  • [ ] Adding hints about into_par_iter() and enumerate() to the documentation for ArrayBase::axis_iter(). Or adding corresponding convenience methods ArrayBase::indexed_axis_iter().
  • [ ] Implementing IntoParallelIterator for IndexedIter. Assuming it can be more efficient than Zip::indexed().
  • [ ] Implementing IntoIterator for Zip.

What are the maintainers' views on these ideas? I would be happy to contribute pull requests with some guidance.

benkay86 avatar Jul 14 '21 17:07 benkay86

Needs further discussion so that the underlying abstractions are clarified - I'll be a bit unreachable during the summer, but back in a bit

bluss avatar Jul 19 '21 00:07 bluss