JuliaSyntax.jl
JuliaSyntax.jl copied to clipboard
Failure to parse some octal escape sequences
julia> Meta.parse(raw""" "\777" """)
"\xff"
julia> JuliaSyntax.parse(Expr, raw""" "\777" """)
ERROR: ArgumentError: octal escape sequence out of range
Stacktrace:
[1] unescape_julia_string(io::IOBuffer, str::SubString{String})
@ JuliaSyntax ~/.julia/packages/JuliaSyntax/OawBx/src/value_parsing.jl:172
[2] unescape_julia_string(str::SubString{String}, is_cmd::Bool, is_raw::Bool)
@ JuliaSyntax ~/.julia/packages/JuliaSyntax/OawBx/src/value_parsing.jl:203
[3] JuliaSyntax.SyntaxNode(source::JuliaSyntax.SourceFile, raw::JuliaSyntax.GreenNode{JuliaSyntax.SyntaxHead}, position::UInt32)
@ JuliaSyntax ~/.julia/packages/JuliaSyntax/OawBx/src/syntax_tree.jl:53
[4] JuliaSyntax.SyntaxNode(source::JuliaSyntax.SourceFile, raw::JuliaSyntax.GreenNode{JuliaSyntax.SyntaxHead}, position::UInt32)
@ JuliaSyntax ~/.julia/packages/JuliaSyntax/OawBx/src/syntax_tree.jl:89
[5] build_tree(::Type{JuliaSyntax.SyntaxNode}, stream::JuliaSyntax.ParseStream; filename::Nothing, kws::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ JuliaSyntax ~/.julia/packages/JuliaSyntax/OawBx/src/syntax_tree.jl:189
[6] build_tree
@ ~/.julia/packages/JuliaSyntax/OawBx/src/syntax_tree.jl:186 [inlined]
[7] #build_tree#90
@ ~/.julia/packages/JuliaSyntax/OawBx/src/expr.jl:208 [inlined]
[8] build_tree
@ ~/.julia/packages/JuliaSyntax/OawBx/src/expr.jl:207 [inlined]
[9] #parse#83
@ ~/.julia/packages/JuliaSyntax/OawBx/src/parser_api.jl:124 [inlined]
[10] parse(::Type{Expr}, input::String)
@ JuliaSyntax ~/.julia/packages/JuliaSyntax/OawBx/src/parser_api.jl:120
[11] top-level scope
@ REPL[81]:1
I think it's the flisp parser which is in error here. \777 is 0x1ff, not 0xff which the flisp parser silently saturates to.
However we shouldn't throw so I think this is still a bug. I think what we need to do here instead is emit an error token and not try to convert to a numeric form.
https://github.com/JuliaLang/JuliaSyntax.jl/pull/83 caused
julia> "\777"
"\xff"
julia> using JuliaSyntax; JuliaSyntax.enable_in_core!()
julia> "\777"
"✘"
We should probably throw a ParseError when encountering an ErrorVal in to_expr. Or just throw that error directly in unescape_julia_string.