JuliaSyntax.jl icon indicating copy to clipboard operation
JuliaSyntax.jl copied to clipboard

Review trivia which can't be inferred from their node kind

Open c42f opened this issue 2 years ago • 4 comments

Some tokens becomes trivia after parsing has done its work. For example, parentheses in both f(x) and (x) only appear in the green tree as trivia. For the function call, the presence of parentheses can be inferred from the tree (call f x) because there's only one type of function call syntax. That is, we can reconstruct the trivia which isn't whitespace or comments.

In contrast, this is not the case for (x), until #222 is merged because the green tree didn't represent grouping parentheses (these ended up as trivia on the parent node instead).

In general, it seems that a nice invariant for the green tree to have would be that

Every piece of syntax can be inferred from the kind, flags, and list of nontrivia children, other than whitespace and comments.

Presumably there's cases other than #222 to fix - would be good to review all cases of bump used together with TRIVIA_FLAG.

c42f avatar Mar 20 '23 11:03 c42f

I looked through parser.jl. General impression is that there's still various cases such as trailing commas etc, but they mostly don't matter.

A borderline case is not being able to distinguish A where B from A where {B} without looking at the trivia. It might be nice to add a flag for that I guess.

c42f avatar Apr 29 '23 19:04 c42f

I don't know, I'm kind of finding myself uninspired by this issue... it seems only marginally useful to change the remaining cases. If we're going to add a flag for A where B does it also makes sense to add one to distinguish things like f(a,b) from f(a,b,)? There's a bunch of such marginal cases and it might not be practical to cover them all vs just dynamically querying the trivia list...

c42f avatar Apr 29 '23 19:04 c42f

Actually let's fix the where case. Not by adding a flag, but by adding an extra tree node to SyntaxNode

c42f avatar Apr 30 '23 23:04 c42f

The where case was fixed in #264. I'm not aware of other cases except for oddities like blocks with empty clauses such as "a ;;; b ;; c" or trailing commas like in (a, b,) which ... idk ... I guess could be useful to have a flag for?

c42f avatar May 07 '23 05:05 c42f