Reduce visitor pain

[erikrose]

trafficstars

It's a pain to write visitors. I can never remember what the formal params are. I want to write a code generator for them, but that's an antipattern. How can we make those go away or shrink?

[mayask]

Could you show an example? Not really getting what you mean

[erikrose]

I think I meant that it's not always obvious even to me, the author, what pieces a rule breaks into for the visitor method's tuple unpack. This is going to be helped somewhat by #29 and #50, and the @rule decorator also helps, for tightly coupled grammar/visitor pairs. But I think there will be an element of documentation that will finally close this ticket.

[erikrose]

Also, let's change the default behavior of some kinds of expressions so we don't need so much boilerplate. For example, OneOf expressions always return a node with one child. There's no point to the outer node, so let's just return the child.

[lucaswiman]

It would reduce coupling if the rules grammar were extended to allow some components to not be passed to visitors. Currently a simple rule like parenthesized_expr = "(" expr ")" needs a visitor like:

    def visit_parenthesized_expr(self, node, (lparen, expr, rparen)):
        return expr

The values passed to visit_ nodes can also be unexpectedly nested. For example, consider the following grammar:

row = value ("," value)*
value = ~"[0-9]+"

I define a visit_value method like:

    def visit_value(self, node, children):
        return int(node.text)

I'd then expect visit_row to receive a list of children like [1, ",", 2, ",", 3] when passed the string 1,2,3. Instead, it gets [1, [[',', 2], [',', 3]]], which is a pretty complicated structure. It also means that the visit_ method is tightly coupled to an implementation detail of the grammar.

One option would be to extend the grammar so parenthesized groupings can optionally be ignored in the visit_methods. For example, [...] could denote a grouping which should stay at the same "level".¹ ^thing would denote grouping that wouldn't be passed to the visit_ methods. So my grammar above would become:

row = value [^"," value]*
value = ~"[0-9]+"

My visit_row method would then get passed a neatly formatted list [1, 2, 3] instead of having to filter out matched delimiters and account for nesting artifacts from the grammar.

If you're agreeable to those changes, I could work on a pull request prototyping them. Thanks for making this amazing library!

^1. My preference would be to make that the default, and only reflect the nesting to visitors if explicitly requested, but I'm guessing that would be a major breaking change for pre-existing NodeVisitor implementations.