Fix path logic on 1.9.0-DEV.725 and newer

[pfitzseb]

Fixes https://github.com/timholy/Revise.jl/issues/691. Fixes https://github.com/timholy/Revise.jl/issues/687.

[jishnub]

This resolves #691 on binaries, but makes things worse on an installation from source. Using the tagged version of Revise, the following works (I'm tracking the source while using Cthulhu)

(@v1.9) pkg> st
Status `~/.julia/environments/v1.9/Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [f68482b8] Cthulhu v2.6.3
  [c3a54625] JET v0.6.0
  [295af30f] Revise v3.3.3

julia> using Cthulhu

julia> @descend sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460
│ ─ %-1  = invoke sin(::Int64)::Float64
1461 1 ─ %1 = Base.sitofp(Float64, x)::Float64                                                                                                                            │╻╷╷ float
1463 2 ─ %2 = invoke Base.Math.sin(%1::Float64)::Float64                                                                                                                  │   
     └──      return %2                                                                                                                                                   │   
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

[ Info: tracking Base
    @eval function ($f)(x::Real)
        xf = float(x)
        x === xf && throw(MethodError($f, (x,)))
        return ($f)(xf)
    end

Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

julia> versioninfo()
Julia Version 1.9.0-DEV.998
Commit e1739aa42a (2022-07-18 10:27 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.5 (ORCJIT, tigerlake)
  Threads: 1 on 8 virtual cores
Environment:
  LD_LIBRARY_PATH = :/usr/lib/x86_64-linux-gnu/gtk-3.0/modules
  JULIA_EDITOR = vim

julia> Revise.juliadir
"/home/jishnu/julia/usr/share/julia/src"

but using this PR

julia> Revise.juliadir
"/home/jishnu/julia/usr/share/julia"

julia> using Cthulhu

julia> @descend sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460
│ ─ %-1  = invoke sin(::Int64)::Float64
1461 1 ─ %1 = Base.sitofp(Float64, x)::Float64                                                                                                                            │╻╷╷ float
1463 2 ─ %2 = invoke Base.Math.sin(%1::Float64)::Float64                                                                                                                  │   
     └──      return %2                                                                                                                                                   │   
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

┌ Error: unable to find path containing source for Base, tracking is not possible
└ @ Revise ~/Dropbox/JuliaPackages/Revise.jl/src/recipes.jl:62
[ Info: tracking Base
    @eval function ($f)(x::Real)
        xf = float(x)
        x === xf && throw(MethodError($f, (x,)))
        return ($f)(xf)
    end┌ Warning: /home/jishnu/julia/usr/share/julia/base/special is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/strings is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/compiler/ssair is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/ryu is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/ is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/docs is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570


Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

julia> @which sin(1)
sin(x::Real)
     @ Base.Math ~/julia/usr/share/julia/src/base/math.jl:1460

This is a strange state, where it actually manages to track Base, but also prints an error indicating otherwise

[pfitzseb]

Does /home/jishnu/julia/usr/share/julia/base/ exist for you? It should be a symlink to /home/jishnu/julia/base/ (unless my build dir is in a weird state).

[jishnub]

/home/jishnu/julia/usr/share/julia/base/ doesn't exist for me.

julia> Revise.juliadir
"/home/jishnu/julia/usr/share/julia"

julia> isdir(Revise.juliadir)
true

julia> isdir(joinpath(Revise.juliadir, "base"))
false

$ ls /home/jishnu/julia/usr/share/julia
base.cache  cert.pem  julia-config.jl  src  stdlib  test

$ ls -dl /home/jishnu/julia/usr/share/julia/src/base
drwxrwxr-x 7 jishnu jishnu 4096 Jul 20 17:27 /home/jishnu/julia/usr/share/julia/src/base

/home/jishnu/julia/usr/share/julia/src/base exists, but it's not a symlink. I wonder if my installation is in an unusual state (although I've not done anything non-standard). Something similar was reported on discourse as well

[pfitzseb]

Ok, that should be fixed now. Turns out the symlink from julia/usr/share/julia/base/ to julia/src/base doesn't exist anymore (I still had it because of an old build).

[jishnub]

Can confirm that this is fixed now. Thank you so much for this!

I have one unrelated question, although I'm unsure if this is a Revise issue or a Cthulhu one:

julia> using Cthulhu

julia> @which sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460

julia> @descend sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460
│ ─ %-1  = invoke sin(::Int64)::Float64
1461 1 ─ %1 = Base.sitofp(Float64, x)::Float64                                                                                                                            │╻╷╷ float
1463 2 ─ %2 = invoke Base.Math.sin(%1::Float64)::Float64                                                                                                                  │   
     └──      return %2                                                                                                                                                   │   
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

[ Info: tracking Base
    @eval function ($f)(x::Real)
        xf = float(x)
        x === xf && throw(MethodError($f, (x,)))
        return ($f)(xf)
    end

Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

julia> @which sin(1)
sin(x::Real)
     @ Base.Math ~/julia/usr/share/julia/src/base/math.jl:1460

The file name display format in the output of @which seems to change after I track the source with Cthulhu. Why is this? Is there type-piracy involved in either of the packages?

[pfitzseb]

Mh, I think that happens because Revise writes the full path into the method table:

julia> @which(gcd(2,3)).file
Symbol("intfuncs.jl")

julia> using Revise

julia> @which(gcd(2,3)).file
Symbol("intfuncs.jl")

julia> Revise.track(Base)

# make some changes to gcd

julia> @which(gcd(2,3)).file
Symbol("/home/pfitzseb/Documents/Git/julia/usr/share/julia/src/base/intfuncs.jl")

No type piracy happening at all.

[jishnub]

I see. Perhaps type-piracy isn't the right word, but I wonder if writing the full path isn't something that should be done by Base in this case? If revise gets the path to the file wrong, it corrupts subsequent introspection, as seen in https://github.com/JuliaDebug/Cthulhu.jl/issues/299. Getting the path wrong will likely be a bug anyway, but the introspection failure may be hard to trace back to Revise.

[pfitzseb]

So you'd want Base to write the full path into the method table? Not sure that's possible without a performance hit on startup. Anyways, this is unrelated to this PR (or did this not happen in the past?).

[jishnub]

It's unrelated to this PR. Maybe I'll discuss this elsewhere. Thanks again for this!

[timholy]

I guess the key question is whether https://github.com/JuliaLang/julia/pull/45441 is up for reconsideration.

[timholy]

With https://github.com/JuliaLang/julia/pull/46420 merged, should we close this?