revise do blocks

[brandonspark]

What: Currently, parsing for do blocks is rather confusing, due to the fact that it makes no distinction in the syntax tree between the arguments to the do, and the actual expressions which follow it.

In addition, it does not actually permit a terminator to follow the do, making it impossible to parse programs which contain do-blocks with nullary arguments denoted by a semicolon. For instance:

f() do ;
  5
end

will not currently parse.

Chiefly, however, the main concern is that the arguments are not separated from the body of the do, making it difficult to actually use the produced tree.

Why: We should allow these cases to be parsed, and make the tree easier to work with.

[brandonspark]

(as far as I'm aware, I've regenerated the parser with the most recent Tree-sitter CLI release, but let me know if this is incorrect @maxbrunsfeld)

[maxbrunsfeld]

I don't believe that will work. See the following Julia program:

Thanks for those examples. I think you're pointing out some real limitations in the way that we parse function parameters in general. But these issues are not specific to do blocks, right? Argument destructuring is something we need to add in both places (functions and do blocks). I'm up for adding support for those in the same PR as this, but I don't think we want to model block parameters as expressions, because that has a very different meaning.

What I would do is extract a _parameter rule from the existing parameter_list rule:

_parameter: $ => choice(
  $.identifier,
  $.spread_parameter,
  $.optional_parameter,
  $.typed_parameter
)

And add a new case to this rule: destructured_parameter (which would contain comma-separated parameters beetween parens).

Then I'd use this in do block. Does that seem reasonable?

[ChrHorn]

Did some experimentation what is allowed syntax wise with parameters. Do parameters are a bit more limited compared to normal function parameters so we probably need an extra block for it.

function / anonymous functions allow

• identifier,
• spread_parameter,
• optional_parameter
• typed_parameter
• tuple_parameter
• keyword_parameter

do parameters allow

• identifier,
• spread_parameter,
• typed_parameter
• tuple_parameter

tuple parameters allow

• identifier,
• spread_parameter,
• typed_parameter
• tuple_parameter
• default_parameter (but only a single parameter per tuple)

[maxbrunsfeld]

Thanks for the investigation @ChrHorn. I still think it would be ok to use one _parameter rule for both (allowing optional parameters in blocks, even though they aren't technically allowed) if it makes the grammar simpler. But if when implementing it, you find it preferable to have two separate parameter rules, then I'm ok with that too.

[savq]

The changes proposed here where incorporated in other PRs (#54 and #83 in particular). There's now a _do_parameter_list rule (but there's not a parameter rule). do-parameters are indeed overly complicated, and it'd be better if they behaved more like arrow function parameters, but it is what it is.