Gensym with shadowing produces incorrect lua

[mrichards42]

I'm not even sure what to title this bug report, but I've come across some funny interactions between gensym, local shadowing and compiler(?) scope.

The piece of code I'm looking at:

(let [x (let [x :inner] x)] x)
;; should result in
;; => "inner"

Ordinarily this emits the following code, which works:

local x
do
  local x2 = "inner"
  x = x2 -- sets the outer x
end

But using gensym I can end up in a situation where this emits the following, incorrect, code:

local x
do
  local x = "inner"
  x = x  -- sets the inner x; outer x is shadowed here
end

A full Fennel example:

;; a few example macros that expand to basically the same thing
(macro not-broken1 [x val]
  `(let [,x (let [,x ,val] ,x)] ,x))

(macro not-broken2 [val]
  `(let [x# (let [x# ,val] x#)] x#))

(macro broken [name val]
  (let [x (gensym (tostring name))]
    `(let [,x (let [,x ,val] ,x)] ,x)))

;; running the expansions manually before using a macro works
(let [x (let [x 1] x)] x)             ; => 1
(let [x_0_ (let [x_0_ 2] x_0_)] x_0_) ; => 2
(let [y_1_ (let [y_1_ 3] y_1_)] y_1_) ; => 3

;; the first two work after expansion; the third does not
(not-broken1 x 1) ; => 1
(not-broken2 2)   ; => 2
(broken y 3)      ; => nil

;; expanding the macros shows that all three are essentially the same
(macrodebug (not-broken1 x 1)) ; => (let [x (let [x 1] x)] x)
(macrodebug (not-broken2 2))   ; => (let [x_0_ (let [x_0_ 2] x_0_)] x_0_)
(macrodebug (broken y 3))      ; => (let [y_1_ (let [y_1_ 3] y_1_)] y_1_)

;; but running the expansions manually again shows that the third does not work!
(let [x (let [x 1] x)] x)             ; => 1
(let [x_0_ (let [x_0_ 2] x_0_)] x_0_) ; => 2
(let [y_1_ (let [y_1_ 3] y_1_)] y_1_) ; => nil

;; introducing an error to see compiled output
(let [y_1_ (let [y_1_ 3] y_1_)] (lua :nope) y_1_)
; [... snip ...]
; local y_1_
; do
;   local y_1_ = 3
;   y_1_ = y_1_    ; <-- oops
; end
; nope
; [... snip ...]

;; compare to the second version, which has never seen a gensym
(let [x_0_ (let [x_0_ 2] x_0_)] (lua :nope) x_0_)
; [... snip ...]
; local x_0_
; do
;   local x_0_0 = 2
;   x_0_ = x_0_0    ; <-- ok
; end
; nope
; [... snip ...]

This same behavior can be observed with (do (var x (do (var x 1) x) x) since it emits the same lua code as the let.

[technomancy]

Is this the same problem as https://todo.sr.ht/~technomancy/fennel/57 ?

[mrichards42]

I imagine they're related, and it might be that the fix is the same. The difference afaict is that https://todo.sr.ht/~technomancy/fennel/57 is about shadowing in the macroexpanded fennel, whereas here I'm intentionally shadowing a symbol, and ending up with incorrect lua when I've used gensym.

[mrichards42]

I can track this down as far as compiler.destructure -> declare-local -> local-mangling -> unique-mangling https://github.com/bakpakin/Fennel/blob/4704eac8a7b741fab2109a9046d3bbcfd887cbe1/src/fennel/compiler.fnl#L100-L103

It seems like a bug to me, but line 101 makes it pretty clear that gensyms aren't supposed to get unique manglings, even if they're declared twice. I assume there's a problem that's solving, so I wouldn't want to undo that and break something else without knowing more about what's going on.

[frenchy64]

I think the fix for https://todo.sr.ht/~technomancy/fennel/54 caused this issue: https://github.com/bakpakin/Fennel/commit/6db6a2caa6e0f2995f749c22c3ee339492799130

FWIW I don't think the general problem of "lua forms respecting local scope" is fixable in all cases using this approach.

e.g.,

>> (fn [] (let [a 1 a 2] (lua "return a")))
Bad code generated - likely a bug with the compiler:
--- Generated Lua Start ---
local function _3_()
  local a = 1
  local a0 = 2
  return a
  return nil
end
pcall(function() require("fennel").metadata:setall(_3_, "fnl/arglist", {}) end)

return _3_--- Generated Lua End ---

It might be worth trying a different approach like this.

[technomancy]

Looks like this is fixed now; thanks @frenchy64!