CSTParser.jl
CSTParser.jl copied to clipboard
Parsing error with very weird use of `<`
ref https://github.com/domluna/JuliaFormatter.jl/issues/465
julia> s = """
function weird()
begin
# notice the trailing "<"
for a in [] <
end
end
println(321)
end
function badly_formatted()
if 2 == 2
println(123)
end
end
weird()
badly_formatted()
"""
"function weird()\n begin\n # notice the trailing \"<\"\n for a in [] <\n end\n end\n println(321)\n end\nfunction badly_formatted()\n if 2 == 2\n println(123)\n end\nend\nweird()\nbadly_formatted()\n"
julia> CSTParser.parse(s)
1:246 function
1:12 call
1:5 weird
13:237 block
13:127 block
13:58 for
13:42 errortoken( CSTParser.InvalidIterator)
13:42 comparison
13:14 a
15:17 OP: in
18:20 vect
21:30 OP: <
31:42 errortoken( CSTParser.UnexpectedToken)
31:42 END
43:42 block
59:75 call
59:65 println
66:68 INTEGER: 321
128:211 function
128:154 call
128:142 badly_formatted
155:198 block
155:198 if
155:165 call
155:157 OP: ==
158:159 INTEGER: 2
160:165 INTEGER: 2
166:191 block
166:191 call
166:172 println
173:175 INTEGER: 123
212:219 call
212:216 weird
220:237 call
220:234 badly_formatted
(JuliaFormatter) pkg> st
Project JuliaFormatter v0.15.9
Status `~/.julia/dev/JuliaFormatter.jl/Project.toml`
[00ebfdb7] CSTParser v3.2.4
[a80b9123] CommonMark v0.8.2
[864edb3b] DataStructures v0.18.9
[0796e94c] Tokenize v0.5.18
[44cfe95a] Pkg
Closing this. While Meta.parse supports this syntax, it also just discards the superfluous token, which I'd prefer not to do.