itertools icon indicating copy to clipboard operation
itertools copied to clipboard

with_ref()?

Open matthiasbeyer opened this issue 3 years ago • 1 comments

I don't know how to properly write this down, so bear with me please.

I constantly have the "issue" that I can chain a lot of functions together, but in the middle or at the end, I need to add something to the iterator for one step.

E.G.:

fn do_something(&self) -> Result<()> {
    let iter = self.inner
          .iter()
          .map(some)
          .map(function)
          .chain(other)          
          .filter(predicate);
    let stdout = std::io::stdout();
    let out = stdout.lock();
    iter.try_for_each(|elem| write!(out, "{}", elem.foo())
}

This is just some example, I know that in practice, you can fetch stdout and stdout.lock() before actually chaining up the iterator because it is lazy - but consider some interfaces where you call not only iterator functions, but chain "normal" functions, maybe add some async-await here and there, add some ? whereever needed, and so on. Allocating stdout before such a long function-chaining block might actually result in overhead because in the error-case, stdout is discarded right away.

But performance and such things to the side (it is not important to me actually).

Much nicer would be an option where I can write something like this:

fn do_something(&self) -> Result<()> {
    self.inner
          .iter()
          .map(some)
          .map(function)
          .chain(other)          
          .filter(predicate);
          .with_ref_to(|| {
              let stdout = std::io::stdout();
              let lock  = stdout.lock();
              (stdout, lock) // probably both because borrowing
          })

          // now passing (stdout, lock) is returned from the `next()` call in the iterator as reference
          .try_for_each(|((_, out), elem)| write!(out, "{}", elem.foo())
}

What do you think? Maybe there is such an adaptor already and I just missed it, or there's even something in the stdlib that can do something like this (fold is not an option, because sometimes I do not want to fold, but map with the object)? If there is, I missed it.


Feel free to edit the title if you can think of something more appropriate :smile: Also: Thanks a lot for making this awesome library! I like it a lot!

matthiasbeyer avatar Jun 22 '21 14:06 matthiasbeyer

The Iterator trait requires the elements to be able to outlive the iterator itself, so you can't just store the result of the closure passed to with_ref_to in the iterator field. You either need Rc or implement IntoIterator for &'a mut WithRefTo and yield references with the 'a lifetime to the data returned by that closure.

However even with those hacks, you still wouldn't be able to rewrite your example because:

  • write requires an exclusive reference to out, but WithRefTo can't give out exclusive references to it, otherwise multiple calls to next would yield multiple exclusive references, which is not possible.
  • You can't return (stdout, lock) from that closure because it would move stdout while lock still borrows from it.

SkiFire13 avatar Jun 28 '21 08:06 SkiFire13

I simply agree with SkiFire13 and I'm therefore closing this.

Philippe-Cholet avatar Jan 12 '24 17:01 Philippe-Cholet