Debugger macroexpansion is not consistent with runtime / Specter Unable to resolve var

[owenRiddy]

Expected behavior

Say I have this code (which is a test case developed from a stacktrace in Spectre which does similar things but more reasonably):

(defmacro d []
  (let [probably-dummy# (-> &env keys first)]
    `(~probably-dummy#)))

(letfn [(dummy [] true)]
   (d))
; Macro `d` locates `dummy` in a roundabout way and calls it
; => true

I expect to be able to instrument that function in the debugger:

#dbg
(letfn [(dummy [] true)]
   (d))
; Macro `d` locates `dummy` in a roundabout way and calls it
; => true

Actual behavior

A stacktrace.

1. Caused by java.lang.IllegalArgumentException
   Can't call nil, form: (nil)

It seems that clojure.walk/macroexpand-all doesn't set up bindings in the same way as a real evaluation would.

Steps to reproduce the problem

This code reproduces the stacktrace.

(defmacro d []
  (let [probably-dummy# (-> &env keys first)]
    `(~probably-dummy#)))

#dbg
(letfn [(dummy [] true)]
   (d))

Environment & Version information

Emacs (Debian testing).

;; CIDER 1.7.0-snapshot (package: 1.7.0-snapshot), nREPL 1.0.0
;; Clojure 1.11.1, Java 17.0.8-ea
;;     Docs: (doc function-name)
;;           (find-doc part-of-name)
;;   Source: (source function-name)
;;  Javadoc: (javadoc java-object-or-class)
;;     Exit: <C-c C-q>
;;  Results: Stored in vars *1, *2, *3, an exception in *e;
;;  Startup: /usr/local/bin/clojure -Sdeps '{:deps {nrepl/nrepl {:mvn/version "1.0.0"} cider/cider-nrepl {:mvn/version "0.30.0"} refactor-nrepl/refactor-nrepl {:mvn/version "3.6.0"}} :aliases {:cider/nrepl {:main-opts ["-m" "nrepl.cmdline" "--middleware" "[refactor-nrepl.middleware/wrap-refactor,cider.nrepl/cider-middleware]"]}}}' -M:cider/nrepl

CIDER version information

;; CIDER 1.7.0-snapshot (package: 1.7.0-snapshot), nREPL 1.0.0
;; Clojure 1.11.1, Java 17.0.8-ea

Lein / Clojure CLI version

Project did have the following deps while constructing the test case, but probably shouldn't be needed to reproduce behavior:

{:deps {com.rpl/specter {:mvn/version "1.1.4"}}}

Emacs version

28.2

Operating system

Debian Testing.

JDK distribution

$ java -version
openjdk version "17.0.8-ea" 2023-07-18
OpenJDK Runtime Environment (build 17.0.8-ea+6-Debian-3)
OpenJDK 64-Bit Server VM (build 17.0.8-ea+6-Debian-3, mixed mode, sharing)

[owenRiddy]

I investigated this issue and it seems to come down to the behaviour of clojure.walk/macroexpand-all. Eg,

(clojure.walk/macroexpand-all
 '(letfn [(dummy [] true)]
   (d)))
; => (letfn* [dummy (fn* dummy ([] true))] (nil))
; Clearly not how the macro actually expands when evaluated

I assume this is some interaction with special forms. It makes spectre-related operations challenging to debug with letfn functions.

I'm not sure how this is best fixed. Since macroexpand-all is not guaranteed to be reliable it may be prudent to document this behaviour at https://docs.cider.mx/cider/debugging/debugger.html as a caveat and if possible provide some sort of #dbg like form that does not macroexpand. I don't need to dig in to the specter internals.