Building a stable and extensible parser API (and hiding the rougher implementation details behind sealed traits)

[zesterer]

Some things me might want to hide:

• Mode
• Check/Emit/etc.
• Parser::go/Parser::go_check/Parser::go_emit/etc.
• InputRef
• Most of Input (Input::next_maybe, for example)

Hiding these details does not necessarily make the crate non-extensible! A viable route to recovering extensibility is to have a more simplistic set of traits for common interfaces (like Parser, Input, etc.) that provide a more limited API for which we can guarantee long-term stability, and then provide blanket implementations of the more complex sealed traits for implementers of the simpler traits.

The goal here is not to hurt downstream developers that may want to build upon chumsky, but to protect them from changes to volatile implementation details that are likely to happen as we find more ways to aggressively optimise parsers. This has a twofold benefit:

1. Developers can extend the trait without fear that the ground is suddenly going to shift under them, providing room for an ecosystem to build up around a stable core interface

2. We, as developers of said core, get the change to aggressively optimise the core and alter the way in which parsers work to provide users with better performance without worrying that we're hurting downstream extensions with regular breaking changes to APIs

How you can help

If you're thinking about building on top of chumsky in the future - perhaps to add new parsers, combinators, errors, etc - then knowing what requirements you have would be enormously helpful! What interfaces would you like to see around backtracking, error generation, token processing, etc.?