Revise.jl icon indicating copy to clipboard operation
Revise.jl copied to clipboard

Revise does not apply functions in the two-argument form of `include`

Open ranocha opened this issue 4 years ago • 27 comments

The two-argument form include(mapexpr::Function, path::AbstractString) allows transforming the code in the file at path. However, Revise does not seem to apply the same transformation mapexpr when re-loading code that is included using this form. For example

julia> using Pkg; Pkg.activate(temp=true); Pkg.add("Revise")
  Activating new environment at `/tmp/jl_D95G2q/Project.toml`
    Updating registry at `~/.julia/registries/General`
    Updating git-repo `https://github.com/JuliaRegistries/General`
   Resolving package versions...
    Updating `/tmp/jl_D95G2q/Project.toml`
  [295af30f] + Revise v3.1.17
[...]

julia> "BrokenModule" |> mkdir |> cd

julia> "src" |> mkdir |> cd

julia> open("BrokenModule.jl", "w") do io
           println(io, """
               module BrokenModule

               include(expr -> quote @fastmath begin \$expr end end, "broken_a.jl")

               end
           """)
       end

julia> open("broken_a.jl", "w") do io
           println(io, """
               foo(a, b, c) = a * b + c
           """)
       end

julia> cd(dirname(pwd()))

julia> open("Project.toml", "w") do io
           println(io, """
               name = "BrokenModule"
               uuid = "79e551b4-d023-11eb-261f-b9feb8f119ff"
               authors = ["Someone"]
               version = "0.0.1"

               [compat]
               julia = "1.6"
           """)
       end

julia> run(`git init`)
Initialized empty Git repository in ~/.julia/dev/BrokenModule/.git/
Process(`git init`, ProcessExited(0))

julia> run(`git add .`)
Process(`git add .`, ProcessExited(0))

julia> run(`git commit -m 'initial commit'`)
[master (root-commit) 78b2121] initial commit
 3 files changed, 16 insertions(+)
 create mode 100644 Project.toml
 create mode 100644 src/BrokenModule.jl
 create mode 100644 src/broken_a.jl
Process(`git commit -m 'initial commit'`, ProcessExited(0))

julia> Pkg.add(url=pwd())
    Updating git-repo `~/.julia/dev/BrokenModule`
   Resolving package versions...
    Updating `/tmp/jl_D95G2q/Project.toml`
  [79e551b4] + BrokenModule v0.0.1 `~/.julia/dev/BrokenModule#master`
    Updating `/tmp/jl_D95G2q/Manifest.toml`
  [79e551b4] + BrokenModule v0.0.1 `~/.julia/dev/BrokenModule#master`

julia> Pkg.develop("BrokenModule")
Path `~/.julia/dev/BrokenModule` exists and looks like the correct package. Using existing path.
   Resolving package versions...
    Updating `/tmp/jl_D95G2q/Project.toml`
  [79e551b4] ~ BrokenModule v0.0.1 `~/.julia/dev/BrokenModule#master` ⇒ v0.0.1 `~/.julia/dev/BrokenModule`
    Updating `/tmp/jl_D95G2q/Manifest.toml`
  [79e551b4] ~ BrokenModule v0.0.1 `~/.julia/dev/BrokenModule#master` ⇒ v0.0.1 `~/.julia/dev/BrokenModule`

julia> using Revise, BrokenModule
[ Info: Precompiling BrokenModule [79e551b4-d023-11eb-261f-b9feb8f119ff]

julia> BrokenModule.foo(1.0, 2.0, 3.0) # This used FMA, as desired when using `@fastmath`
5.0

julia> @code_native debuginfo=:none syntax=:intel BrokenModule.foo(1.0, 2.0, 3.0)
        .text
        vfmadd213sd     xmm0, xmm1, xmm2        # xmm0 = (xmm1 * xmm0) + xmm2
        ret
        nop     word ptr cs:[rax + rax]

julia> open(joinpath(dirname(pathof(BrokenModule)), "broken_a.jl"), "w") do io
           println(io, """
               foo(a, b, c) = a + b * c
           """)
       end

julia> BrokenModule.foo(1.0, 2.0, 3.0) # This doesn't use FMA although we have `@fastmath`
7.0

julia> @code_native debuginfo=:none syntax=:intel BrokenModule.foo(1.0, 2.0, 3.0)
        .text
        vmulsd  xmm1, xmm1, xmm2
        vaddsd  xmm0, xmm1, xmm0
        ret
        nop     dword ptr [rax]

The expected output of the last line would be something like

julia> @fastmath foo(a, b, c) = a + b * c
foo (generic function with 1 method)

julia> @code_native debuginfo=:none syntax=:intel foo(1.0, 2.0, 3.0)
        .text
        vfmadd231sd     xmm0, xmm2, xmm1        # xmm0 = (xmm2 * xmm1) + xmm0
        ret
        nop     word ptr cs:[rax + rax]

Hence, this demonstrates that Revise did not apply the transformation expr -> quote @fastmath begin $expr end end passed to include.

ranocha avatar Jun 18 '21 11:06 ranocha

Is this an issue that seems easily fixable or is this a tougher nut to crack? It would be really nice to be able to use the 2-argument include with Revise :-)

sloede avatar Jun 25 '21 08:06 sloede

It's not trivial, but not impossible either. Someone would need to modify Revise's internal data structures to keep track of a mapexpr for each source file. I see some links to Trixi above, is that the only package that uses this feature?

timholy avatar Jul 18 '21 10:07 timholy

We wanted to use this feature of include but discovered the issue described above while experimenting with it. Our current workaround is to basically wrap each file in a macro to get the same results while Revise works fine. It seems to be more clean if we could use the two-argument form of include instead.

ranocha avatar Jul 18 '21 10:07 ranocha