doomemacs
doom binary utility fails constantly
I confirm that...
-
[X] I have searched the issue tracker, documentation, FAQ, Discourse, and Google, in case this issue has already been reported/resolved.
-
[X] I have read "How to Debug Issues", and will use it to provide as much information about this issue as possible.
-
[X] The issue can be reproduced on the latest available commit of Doom.
-
[X] The issue can be reproduced on a stable release of Emacs, such as 27 or 28. (Doom does not support development builds like 29+ or any version ending in .50 or .9x)
Expected behavior
When I use the doom binary utility, it should work with any command.
Current behavior
What happens instead is if I use doom with any command (Or even no command at all) it outputs the same error message:
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Symbol’s value as variable is void:
Obviously, no pastebin on this one.
Steps to reproduce
- Clone the repo
~/.emacs.d/bin/doom
System Information
error
Please tell me about your system:
- Operating System and version.
- Installed version of Emacs (and how it was installed).
- Where Doom was cloned.
- Are you using Emacs with flatpak or snap?
Stems from native-comp. Try emacs without native-comp...
Related to https://github.com/doomemacs/doomemacs/issues/5217
A commonality in the reports I'm seeing is that they come from users who have installed Emacs (with native-comp) with Snap or Flatpak. Can either of you confirm?
I'm using neither Snap nor Flatpak, just using the Solus provided Emacs.
No clue how to confirm whether or not I have native-comp.
Hi,
I have the same error, running
- MacOS 11
- Emacs 28.2.50 built from source using https://github.com/jimeh/build-emacs-for-macos
- Yes, native-comp is enabled (I consider it a neccessity these days.).
- GCC and libgccjit are install via brew on v12.2.0
Emacs it self starts fine using -q and before I removed .local, it worked well even with
my doom config. So I assume the CLI does something different?!
Is there something I can do to corner this?
So, printf-debugging shows that this block in doom fails
(condition-case e
(let* ((bin-dir (file-name-directory (file-truename load-file-name)))
(init-file (expand-file-name "../early-init.el" bin-dir)))
(or (and (load init-file nil 'nomessage 'nosuffix)
(featurep 'doom))
(user-error "Failed to load Doom from %s" init-file)))
;; Prevent ugly backtraces for trivial errors
(user-error (message "Error: %s" (cadr e))
(kill-emacs 2)))
Digging in a bit deeper
load-file-nameisnilbin-dirfunction definition is void
doom only gets this far when (setq load-prefer-newer nil)
is set before, otherwise
load-with-code-conversion: Recursive load: "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/help-fns.el.gz"
which has become a constant issue since native comp was enabled.
@hlissner any chance of progress, or is this too specific?
load-file-name is nil
This isn't consistent with the errors reported. If load-file-name were nil, file-truename (or something inside it) should throw a type error, but OP reports void-variable (i.e. undefined variable) errors. Unless I misunderstood, you're reporting the same, along with recursive load errors). None of these are type errors.
bin-dir function definition is void
bin-dir is a let-binding, not a function call. If you are seeing this error, it sounds strangely like a byte-compilation issue. The kind I'd see if bin/doom itself were byte-compiled with a version of Emacs so old that it didn't have let* defined yet (which I don't think exists; let* must be older than Emacs 18, at least).
In any case, Doom doesn't byte-compile its files (yet), so something is up. Have you byte-compiled it (or Doom's source files) yourself?
Maybe you are running bin/doom with a different version of Emacs than the one you are launching it with? Try to explicitly set the executable:
EMACS=/path/to/correct/emacs doom sync
(setq load-prefer-newer nil)
The recursive load error seems unrelated to this issue, but can you produce a backtrace from it?
Stems from native-comp. Try emacs without native-comp...
Related to https://github.com/doomemacs/doomemacs/issues/5217
This issue is not related to #5217. While these issues seem to be related to native-comp builds, they aren't caused by native compilation, aren't the kind of errors native compilation would produce, and the errors reported here and in #5217 are nothing alike.
@Qfl3x Does running bin/doom in debug mode produce a backtrace? I.e.
DEBUG=1 doom sync
Well, the error at the commandline is the same as reported by @Qfl3x. Inserting (message "I am here") after every
top-level evaluation showed that it is thrown in the first form (condition-case .... Interestingly, the error message
is not formatted like the handler suggests ("Error: ...").
I have some LISP knowledge, but none specific to ELISP, so bear with my ignorance.
Recursive load
What I can see from the builds script I used, it AOT-compiles some .el, compresses, and embeds them into Emacs.app.
Native comp
It works when using the emacs-mac via homebrew including native-comp.
I have run into similar issue today.
some of the output ⬇️
Eager macro-expansion failure: (void-variable DLL)
Eager macro-expansion failure: (void-variable DLL)
Eager macro-expansion failure: (void-variable DLL)
Eager macro-expansion failure: (void-variable DLL)
Debugger entered--Lisp error: (void-variable DLL)
byte-code("\10\210\11\210\n\210\10\210\11\210\n\210\303\304!\207" [DLL init: proxychains-ng require macroexp] 2)
(pcase (pop rest) (:append (setq append-p t)) (:depth (setq depth (pop rest))) (:local (setq local-p t)) (:remove (setq remove-p t)))
(while (keywordp (car rest)) (pcase (pop rest) (:append (setq append-p t)) (:depth (setq depth (pop rest))) (:local (setq local-p t)) (:remove (setq remove-p t))))
(let* ((hook-forms (doom--resolve-hook-forms hooks)) (func-forms nil) (defn-forms nil) append-p local-p remove-p depth) (while (keywordp (car
the machine i'm using is MacBook Pro (13-inch, 2020, Four Thunderbolt 3 ports)
what did I do
- uncomment
;;(emoji +unicode)in~/.doom.d/init.el - run
proxychains4 emacs, once it complete its download, then exit - try run
~/.emacs.d/bin/doom syncagain
why I use proxychains: cuz my country has very weak connection to githubusercontent.com for some reason 😭
How do I solve
I just delete ~/.emacs.d/eln-cache dir, then I try ~/.emacs.d/bin/doom sync, things works fine
reproduce
but when try to reproduce this issue by the following steps
- run
proxychains4 emacsand exit ~/.emacs.d/bin/doom sync
the output has changed to this
Error: failed to run Emacs with command 'emacs'
Are you sure Emacs is installed and in your $PATH?
This can also be solved by just delete ~/.emacs.d/eln-cache, don't know if it's the same issue
This seems like the most active thread that is most closely related to this issue I'm having, so I'm going to share my notes here. Apple M2 Max on Ventura 13.4. I've tried both the latest doomemacs commit and a random commit from November of last year. Same behavior.
➜ emacs git:(master) ✗ ./bin/doom install
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Symbol’s value as variable is void:
I'm running emacs-28.1.91 with CONFIG_SHELL=/bin/sh /bin/sh ./configure --with-cairo --with-xwidgets --with-mailutils. I also tried emacs-28.2 and have this exact same issue. I was first experiencing this with --with-native-comp but after reading this I tried without native compilation and experienced the same issue. All pastes below come from a build without native compilation. I'll try to share how I verified that I was actually running the non-native-comp binary so that there's no mistake that I accidentally forgot to link something or had the wrong binary in my $PATH. But don't take my word for it; if you can think of something else I need to verify, please let me know.
➜ emacs git:(master) ✗ emacs --version
GNU Emacs 28.1.91
➜ emacs git:(master) ✗ emacs --batch --eval "(message (describe-variable 'system-configuration-features))"
system-configuration-features is a variable defined in ‘C source code’.
Its value is
"ACL GNUTLS LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS PDUMPER PNG THREADS TOOLKIT_SCROLL_BARS XWIDGETS ZLIB"
...
The exact bytes of the void variable appear as blank spaces in my terminal, a non-displayable glyph in my browser, and have the byte value c28b.
➜ emacs git:(master) ✗ ./bin/doom install 2>&1 | xxd
00000000: 4561 6765 7220 6d61 6372 6f2d 6578 7061 Eager macro-expa
00000010: 6e73 696f 6e20 6661 696c 7572 653a 2028 nsion failure: (
00000020: 766f 6964 2d76 6172 6961 626c 6520 c28b void-variable ..
00000030: 290a 4561 6765 7220 6d61 6372 6f2d 6578 ).Eager macro-ex
00000040: 7061 6e73 696f 6e20 6661 696c 7572 653a pansion failure:
00000050: 2028 766f 6964 2d76 6172 6961 626c 6520 (void-variable
00000060: c28b 290a 5379 6d62 6f6c e280 9973 2076 ..).Symbol...s v
00000070: 616c 7565 2061 7320 7661 7269 6162 6c65 alue as variable
00000080: 2069 7320 766f 6964 3a20 c28b 0a is void: ...
I added some print statements in doom to see if I could find the last line that gets run before the error message.
;; Doom's core sets up everything we need; including `doom-*-dir' variables,
;; universal defaults, and autoloads for doom-*-initialize functions.
(print "line 83 in doom")
(condition-case e
(let* ((bin-dir (file-name-directory (file-truename load-file-name)))
(init-file (expand-file-name "../early-init.el" bin-dir)))
(or (and (load init-file nil 'nomessage 'nosuffix)
(featurep 'doom))
(user-error "Failed to load Doom from %s" init-file)))
;; Prevent ugly backtraces for trivial errors
(user-error (message "Error: %s" (cadr e))
(kill-emacs 2)))
(print "line 94")
I see "line 83" print but I don't see "line 94" print.
I continued adding print statements until I narrowed it down to (eval-when-compile (require 'subr-x)) in lisp/doom.el.
;;; Code:
(print "pre subr-x")
(eval-when-compile (require 'subr-x))
(print "post subr-x")
;;; Version checks
➜ emacs git:(master) ✗ ./bin/doom install
"pre-subr-x"
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Symbol’s value as variable is void:
I don't know enough about lisp to know what exactly eval-when-compile is doing, but I was curious what would happen if I removed it. It just gets rid of the Eager macro-expansion failure message and leaves me with the Symbol's value as variable is void message.
➜ emacs git:(master) ✗ ./bin/doom install
"pre-subr-x"
Symbol’s value as variable is void:
Anyways... I guess now it's breaking somewhere in subr-x.el?
I found subr-x.elc and subr-x.el.gz in /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/. I unzipped subr-x.el.gz, added a print statement at the very beginning, just before the very first significant line of code (eval-when-compile (require 'cl-lib)), ran emacs --batch --eval "(byte-compile-file \"subr-x.el\")", and re-ran doom install. (I don't even know if this is a valid way to re-compile .elc files in this directory. I'm really aiming blindly here.)
➜ emacs git:(master) ✗ ./bin/doom install
"pre-subr-x"
"about to compile cl-lib"
"post-subr-x"
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Symbol’s value as variable is void:
Now I get past subr-x and fail somewhere else.
If anyone wants to collaborate, I'm happy to donate my broken environment and my time to figuring this out. I'm way past my knowledge and I don't know which paths would be useful to go down and which are just noise.
The recursive load error seems unrelated to this issue, but can you produce a backtrace from it?
I've been unsuccessful in producing a backtrace.
DEBUG=1 EMACS=/usr/local/bin/emacs ./bin/doom install where cat /usr/local/bin/emacs is:
#!/bin/sh
/Applications/Emacs.app/Contents/MacOS/Emacs --debug-init "$@"
... doesn't yield anything related to a Doom log.
➜ emacs git:(master) ✗ find ~ -type d -name logs
/Users/eihli/.config/emacs/.git/logs
...
➜ emacs git:(master) ✗ find ~ -iregex '.*doom.*log.*'
find: /Users/eihli/Library/Application Support/CallHistoryTransactions: Operation not permitted
# etc...
I just noticed this "cycle detection" comment in macroexp.el, so maybe the recursive load error is related? (I'm only drawing that conclusion because I associate "cycle dection" with "recursion" and not because I understand what's going on behind the scenes.)
(error
;; Hopefully this shouldn't happen thanks to the cycle detection,
;; but in case it does happen, let's catch the error and give the
;; code a chance to macro-expand later.
(message "Eager macro-expansion failure: %S" err)
form))))
Another experimental observation that doesn't yet mean anything to me but I think could be relevant:
In lisp/doom.el, I added some print statements to see which lines run and I saw a comment related to hacky things and noticed I stopped getting (print ...) output around the code that mucks with file-name-handler-alist.
;;; Startup optimizations
(print "Startup optimizations")
;; Here are Doom's hackiest (and least offensive) startup optimizations. They
;; exploit implementation details and unintended side-effects, and will change
;; often between major Emacs releases. However, I disable them if this is a
;; daemon session (where startup time matters less).
;;
;; Most of these have been tested on Linux and on fairly fast machines (with
;; SSDs), so your mileage may vary depending on your hardware.
(unless (daemonp)
;; PERF: `file-name-handler-alist' is consulted on each call to `require',
;; `load', or various file/io functions (like `expand-file-name' or
;; `file-remote-p'). You get a noteable boost to startup time by unsetting
;; or simplifying its value.
(let ((old-value (default-toplevel-value 'file-name-handler-alist)))
(print "this runs")
(setq file-name-handler-alist
;; HACK: If the bundled elisp for this Emacs install isn't
;; byte-compiled (but is compressed), then leave the gzip file
;; handler there so Emacs won't forget how to read read them.
;;
;; calc-loaddefs.el is our heuristic for this because it is built-in
;; to all supported versions of Emacs, and calc.el explicitly loads
;; it uncompiled. This ensures that the only other, possible
;; fallback would be calc-loaddefs.el.gz.
(if (eval-when-compile
(locate-file-internal "calc-loaddefs.el" load-path))
nil
(list (rassq 'jka-compr-handler old-value))))
;; Make sure the new value survives any current let-binding.
(set-default-toplevel-value 'file-name-handler-alist file-name-handler-alist)
;; Remember it so it can be reset where needed.
(put 'file-name-handler-alist 'initial-value old-value)
;; COMPAT: ...but restore `file-name-handler-alist' later, because it is
;; needed for handling encrypted or compressed files, among other things.
(add-hook! 'emacs-startup-hook :depth 101
(defun doom--reset-file-handler-alist-h ()
(setq file-name-handler-alist
;; Merge instead of overwrite because there may have been changes to
;; `file-name-handler-alist' since startup we want to preserve.
(delete-dups (append file-name-handler-alist old-value))))))
(print "this doesn't")
I looked in /Applications/Emacs.app/Contents/Resources/lisp/calc and saw that I had a calc-loaddefs.el file. I gzipped it to calc-loaddefs.el.gz and deleted calc-loaddefs.el and re-ran ./bin/doom install.
The result was that I still got the Eager macro-expansion failure error messages but the Symbol's value as variable is void message changed to:
Recursive load: "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/cl-lib.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/debug.elc", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz", "/Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/pcase.el.gz", "/Users/eihli/.config/emacs/lisp/doom.el", "/Users/eihli/.config/emacs/early-init.el", "/Users/eihli/.config/emacs/bin/doom"
...which is the exact same error as https://github.com/doomemacs/doomemacs/issues/681. But in this case, setting load-prefer-newer to nil didn't change any behavior.
I just ran:
gzip -d jka-compr.el.gz
emacs --batch --eval "(byte-recompile-directory (expand-file-name \"/Applications/Emacs.app/Contents/Resources/lisp/\") 0)"
And then re-ran doom install. It looks like it's working.
cli.doom.230612135732.47467.log
I'm going to start from a blank slate on both Emacs and Doom and try this one change again.
Update:
From a blank slate:
git checkout emacs-28.1.91
./autogen.sh
CONFIG_SHELL=/bin/sh /bin/sh ./configure --with-cairo --with-imagemagick --with-xwidgets --with-native-compilation --with-mailutils --no-create --no-recursion
make -j12
make install
cp -r nextstep/Emacs.app /Applications
cat /usr/local/bin/emacs
#!/bin/sh
/Applications/Emacs.app/Contents/MacOS/Emacs --debug-init "$@"
git clone --depth 1 https://github.com/doomemacs/doomemacs ~/.config/emacs
~/.config/emacs/bin/doom install
Eager macro-expansion failure: (void-variable �)
Eager macro-expansion failure: (void-variable �)
Symbol’s value as variable is void: �
gzip -d /Applications/Emacs.app/Contents/Resources/lisp/jka-compr.el.gz
~/.config/emacs/bin/doom install
emacs --batch --eval "(byte-recompile-directory (expand-file-name \"/Applications/Emacs.app/Contents/Resources/\") 0)"
~/.config/emacs/bin/doom install
# ... Fail
gzip -d /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/macroexp.el.gz
emacs --batch --eval "(byte-recompile-directory (expand-file-name \"/Applications/Emacs.app/Contents/Resources/\") 0)"
~/.config/emacs/bin/doom install
# ... Fail
gzip -d /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/subr-x.el.gz
emacs --batch --eval "(byte-recompile-directory (expand-file-name \"/Applications/Emacs.app/Contents/Resources/\") 0)"
~/.config/emacs/bin/doom install
# ... Success
[edit: Well darnit. I just ran these steps again to double check and now it's still failing with the Eager macro-expansion error. My environment either wasn't entirely clean on the first run, or it wasn't entirely clean on the second run, or I typoed something or did something slightly out of order, or this error/fix is non-deterministic. Bah!]
Hey. I'm on windows and I built emacs 28.2 from source with ./configure --without-pop --with-native-compilation and I renamed my .doom.d and cloned doomemacs to .config/emacs so that it's all as fresh as can be.
Running DEBUG=1 doom gives me
Loading c:/Users/me/.config/emacs/lisp/doom.el (source)...
Eager macro-expansion failure: (void-variable )
Eager macro-expansion failure: (void-variable )
Symbol's value as variable is void:
edit: I just built emacs ./configure --without-pop and the error went away.
edit2: It stopped working right after I ran pacman -Syu. I was trying to install gtk3 things because I wanted to build --with-xwidgets...
Another bit of perhaps useful information: when I set EMACS=~/emacs/src/temacs then doom install succeeds, I can open Emacs, and I don't see any error messages. But then, when I'm inside Emacs and I M-x doom/reload, then I get the Eager macro-expansion failure: (void-variable ) message again.
So maybe this issue has something to do with how the pdump is created? https://www.gnu.org/software/emacs/manual/html_node/elisp/Building-Emacs.html
I have a disappointing update.
I put this aside for about a month because I had work to do. I just installed Emacs from Homebrew and went about my day-to-day.
Then, yesterday, I unlinked my brewed version, checked out Emacs 28.1, ran a make bootstrap-clean, ./autogen.sh, CC=clang CFLAGS="-g -O3 -DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT" CXXFLAGS="-g -O3" ./configure --with-cairo --with-imagemagick --with-xwidgets --with-native-compilation --with-mailutils --no-create --no-recursion --with-ns --with-json, make -j12, sudo make install, ./bin/doom install, and...
It worked...
No idea why. I followed the same procedure I remember following last month.