Explain the meaning of eoi in the Stream constructors

[lbfalvy]

I don't really see why this parameter is needed, and as far as I'm aware the end of a set of disjoint ranges is a number and not a range, therefore I can't really figure out what value I should give to this parameter. Should it be the first position outside the input and the consecutive number, as in the nano_rust example? If the +1 comes from the fact that the lexer works on 1-wide characters, under what circumstances would the value be different and what would it be? If the +1 is invariant, why doesn't the stream just store the first invalid index as a literal? I'd be happy to go through the code and update relevant doc comments so that rookies like me can understand it if someone explains to me what this parameter does.

[zesterer]

Hello! The eoi span is required so that chumsky knows what span to produce for zero-width inputs. For example, the parser a.repeated() can accept no input tokens at all (because zero repetitions is valid), and chumsky needs to know what span to assign this input (i.e: in a.repeated().map_with_span(|_, span| ...)).

In general, it should be the span of the end of the input, which most users interpret as a zero-width span starting immediately after the last input token's span.

I'll endeavour to document this a bit better.