JuliaSyntax.jl icon indicating copy to clipboard operation
JuliaSyntax.jl copied to clipboard

Incorrect parsing of `function` with parens.

Open oscardssmith opened this issue 2 years ago • 2 comments

master:

julia> dump(:(function (f(::T) where {T}) end))
Expr
  head: Symbol function
  args: Array{Any}((2,))
    1: Expr
      head: Symbol where
      args: Array{Any}((2,))
        1: Expr
          head: Symbol call
          args: Array{Any}((2,))
            1: Symbol f
            2: Expr
              head: Symbol ::
              args: Array{Any}((1,))
                1: Symbol T
        2: Symbol T
    2: Expr
      head: Symbol block
      args: Array{Any}((2,))
        1: LineNumberNode
          line: Int64 1
          file: Symbol REPL[11]
        2: LineNumberNode
          line: Int64 1
          file: Symbol REPL[11]

JuliaSyntax

julia> dump(:(function (f(::T) where {T}) end))
Expr
  head: Symbol function
  args: Array{Any}((2,))
    1: Expr
      head: Symbol tuple
      args: Array{Any}((1,))
        1: Expr
          head: Symbol where
          args: Array{Any}((2,))
            1: Expr
              head: Symbol call
              args: Array{Any}((2,))
                1: Symbol f
                2: Expr
            2: Symbol T
    2: Expr
      head: Symbol block
      args: Array{Any}((2,))
        1: LineNumberNode
          line: Int64 1
          file: Symbol REPL[21]
        2: LineNumberNode
          line: Int64 1
          file: Symbol REPL[21]

oscardssmith avatar Oct 04 '22 21:10 oscardssmith

Definite inconsistency, thanks for reporting it.

How did you discover this bug and do we actually want to support brackets here?

I ask partly because the reference parser also allows constructs such as

function (((((f(::T) where {T})))))
end

Which seems like bad syntax we probably shouldn't support.

c42f avatar Oct 07 '22 03:10 c42f

The issues I've found were found by trying to precompile a bunch of packages that I use (I forget exactly which one this is, but it's in the SCIML or Deep Learning stack). The reason the parentheses were used here is that the where {T} was on a separate line, so it was intended to make it more clear what the where referred to.

oscardssmith avatar Oct 07 '22 04:10 oscardssmith

So the messy part of this is that the reference parser has some bugs which I've fixed, but in a way which disallows this particular syntax. So now trying to fix this in addition to not breaking the other things I've already fixed is a bit of a pain.

Forms we need to support:

# Normal named functions
function f(x,y)
function (:)(x,y)  # case where parens are necessary for function name
function Base.:+(x,y)   # module paths
function (f(x,y) where T)  # this bug report

# Anon functions
function (x,y)
function (f)
function (x;y)  # y is keyword argument, not part of a block - reference parser gets this wrong
function (x...)   # reference parser produces (... x)  not (tuple (... x))

c42f avatar Oct 19 '22 02:10 c42f

yeah. It's somewhat unfortunate how complex Julia's syntax is. This however is a fairly important one to support because autogenerated code will often have extra random parentheses to make sure there aren't parsing errors.

oscardssmith avatar Oct 19 '22 02:10 oscardssmith

This issue seems to cause an issue when compiling Unitful.jl with JuliaSyntax.jl, https://github.com/PainterQubits/Unitful.jl/blob/242f5c76958dae03c80f6d336780939eccf27edd/src/logarithm.jl#L266-L278

aviatesk avatar Oct 26 '22 10:10 aviatesk

(that is how I found it).

oscardssmith avatar Oct 26 '22 12:10 oscardssmith

Yup, it's tracked in the JuliaSyntax.jl bug tracker and it's one of the next on the list to fix.

c42f avatar Oct 26 '22 23:10 c42f

Ugh, so related to this, it turns out some packages also use pointless syntax like

function (f()) where T
    body
end

Fix in https://github.com/JuliaLang/JuliaSyntax.jl/pull/140 🤦‍♀️

c42f avatar Nov 02 '22 09:11 c42f