enable using a tokenized input as a Stream input

[cosmicexplorer]

Problem

I'm parsing a string into tokens with chumsky, and I would like to also use chumsky to parse those tokens into something else. While select! { ... } is intended to enable this, it assumes that the stream of tokens is produced externally to chumsky, as in the logos example: https://github.com/zesterer/chumsky/blob/56762fe5c965880066a068038a18b5ac07afd75c/examples/logos.rs#L130-L145

Solution

• Expose .parse_iter() outside of #[cfg(test)] and use it to construct a Stream instance.
• Expose .stream(input) as a public method of IterParser to generate a stream of transformed input.

[zesterer]

One thing I worry about is that the API seems to imply that the parser gets turned into a Stream, when in reality it's used used to parse elements, collected into a vector, and then those elements are used as a Stream. #399 discusses the former use-case and what problems we've run up against when trying to do this.

Did you have an example of the sort of patterns that this enables?

[zesterer]

Edit: It seems I misread the implementation earlier, I see it is turning the parser directly into a stream. As mentioned, #399 discusses some of these issues. In particular, ParseIter currently just swallows parser errors, pretending they don't exist.

[cosmicexplorer]

Ah, I see #399 (mentioned directly above parse_iter()) covers exactly this issue, not a different one. I'll see if I can page into that.

[cosmicexplorer]

Converted this into a draft as this is really just the easier part of #399.