chumsky
chumsky copied to clipboard
Allow `repeated` to yield an `Iterator` rather than collecting into a `Vec`
This would allow to skip Stream::from_iter
dance when using Chumsky as lexer as well as parser.
I don't think that yielding in general is possible (at least, without dropping a lot of performance on the floor and increasing the complexity of the crate massively). That said, I think it should be possible to have a trait IterParser<I, O>: Parser<I, O>
extension trait with a method like parse_iter
for specific types like Repeated
and SeparatedBy
.
Figuring out exactly how error propagation should work is still an issue too: an iterator producing (Vec<Self::Error>, Result<O, Self::Error>)
is probably not particularly useful for common use-cases like producing an iterator of tokens, so we'd need to come up with some other 'side effect' channel where errors can be generated. Perhaps parse_iter
collects errors internally and allows fetching them afterward? This might be more ergonomic, but also not well suited to some use-cases.
This is eventually going to be possible with all the bells & whistles via #270, closing in anticipation.