Emacs@{29,30}: Build errors related to protable dump files and fingerprint mismatches

[tzz]

Please make sure to follow these steps (and mark the checkboxes):

• [x] run brew update and try to reproduce the issue again
• [x] run brew doctor, fix all issues and try to reproduce your issue again
• [x] run brew config and brew doctor and include their output

What you were trying to do

I have a problem building @29. I tried with no extra options as well, same error. @30 has the same error as well. This is on a M1. @28 works. I couldn't figure out how to break into the build process to look at fingerprints and other info. I also don't know if the CI actions cover M1 or just Intel architecture.

What happened (include command output)

❯ cat ~/Library/Logs/Homebrew/emacs-plus@29/00.options.out
d12frosted/emacs-plus/emacs-plus@29 --with-imagemagick --with-native-comp --with-no-frame-refocus

Command output

cat ~/Library/Logs/Homebrew/emacs-plus@29/03.gmake
...
Dumping under the name bootstrap-emacs.pdmp
Dumping fingerprint: d4dc2c48fa9f82abbf5ffcaa17058f8dfa1a4edeccb5ba3ba63879aa362cba7f
Dump complete
Byte counts: header=100 hot=13652772 discardable=195472 cold=6890440
Reloc counts: hot=796163 discardable=5352
...
just a little later in the build process (the error repeats with the same fingerprint mismatch):
...
'../src/emacs' -batch --no-site-file --no-site-lisp --eval "(setq load-prefer-newer t byte-compile-warnings 'all)"  

-l comp -f batch-byte+native-compile org/ox.el
desired fingerprint: d4dc2c48fa9f82abbf5ffcaa17058f8dfa1a4edeccb5ba3ba63879aa362cba7f
found fingerprint: dd0a836822fb1c9fa3987e59d656133b927ff0575ec9b0b4ca5b1658f987634f
emacs: could not load dump file "/private/tmp/emacs-plusA29-20230218-33549-301jtr/src/emacs.pdmp": not built for this Emacs executable

Output of brew config

$ brew config
HOMEBREW_VERSION: 4.0.1
ORIGIN: https://github.com/Homebrew/brew
HEAD: 17c872fb5275d87922a56416587cb439a5064354
Last commit: 3 days ago
Core tap origin: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 0f87d15ad2a094b10d662ecdaea3c944272804ff
Core tap last commit: 25 hours ago
Core tap branch: master
Core tap JSON: 19 Feb 05:41 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: vim
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.8 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/2.6.8_1/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 14.0.0 build 1400
Git: 2.39.0 => /opt/homebrew/bin/git
Curl: 7.86.0 => /usr/bin/curl
macOS: 13.2-arm64
CLT: 14.2.0.0.1.1668646533
Xcode: N/A
Rosetta 2: false

Output of brew doctor

$ brew doctor

==> Downloading https://formulae.brew.sh/api/formula.json
-=#=#   #   #
Warning: Calling plist_options is deprecated! Use service.require_root instead.
Please report this issue to the yqrashawn/goku tap (not Homebrew/brew or Homebrew/homebrew-core), or even better, submit a PR to fix it:
  /opt/homebrew/Library/Taps/yqrashawn/homebrew-goku/Formula/goku.rb:16

Warning: Calling plist_options is deprecated! Use service.require_root instead.
Please report this issue to the dimentium/autoraise tap (not Homebrew/brew or Homebrew/homebrew-core), or even better, submit a PR to fix it:
  /opt/homebrew/Library/Taps/dimentium/homebrew-autoraise/autoraise.rb:29

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  unox

[d12frosted]

Don't remember if CI uses M1 or not, but I just tested on M1 and everything worked.

Please try the following:

1. rm -rf ~/Library/Caches/Homebrew/emacs-plus@29--git (maybe even 30 if that exists).
2. Build without any options. If it fails, share the output in this thread.
3. Reinstall tree-sitter (e.g. 'brew uninstall tree-sitterandbrew install tree-sitter). Don't use reinstall` command.
4. Build without any options. If it fails, share the output in this thread.

And please, share the output of brew doctor.

[tzz]

❯ rm -rf ~/Library/Caches/Homebrew/emacs-plus@29--git
❯ rm -rf ~/Library/Caches/Homebrew/emacs-plus@30--git
❯ brew uninstall tree-sitter; brew install tree-sitter
==> Downloading https://formulae.brew.sh/api/cask.json
######################################################################## 100.0%
==> Downloading https://formulae.brew.sh/api/formula.json
######################################################################## 100.0%
Warning: Calling plist_options is deprecated! Use service.require_root instead.
Please report this issue to the yqrashawn/goku tap (not Homebrew/brew or Homebrew/homebrew-core), or even better, submit a PR to fix it:
  /opt/homebrew/Library/Taps/yqrashawn/homebrew-goku/Formula/goku.rb:16

Warning: Calling plist_options is deprecated! Use service.require_root instead.
Please report this issue to the dimentium/autoraise tap (not Homebrew/brew or Homebrew/homebrew-core), or even better, submit a PR to fix it:
  /opt/homebrew/Library/Taps/dimentium/homebrew-autoraise/autoraise.rb:29

Uninstalling /opt/homebrew/Cellar/tree-sitter/0.20.7... (18 files, 12.8MB)
==> Fetching tree-sitter
==> Downloading https://ghcr.io/v2/homebrew/core/tree-sitter/manifests/0.20.7-1
Already downloaded: /Users/tzlatanov/Library/Caches/Homebrew/downloads/10c92a2348e106d4cb1e0c4701c23a9de60150036ccf6c4be88a5a296dc3a8ed--tree-sitter-0.20.7-1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/tree-sitter/blobs/sha256:c1fb86aa4f3e621a51b8a3162ffab2acc64ca814f3cd758bf5e167128a90c8dd
Already downloaded: /Users/tzlatanov/Library/Caches/Homebrew/downloads/0cf12eacb2af0d5690c0b07c15e44cc13c0ac3922179934ae3edd6844bf1929d--tree-sitter--0.20.7.arm64_ventura.bottle.1.tar.gz
==> Pouring tree-sitter--0.20.7.arm64_ventura.bottle.1.tar.gz
🍺  /opt/homebrew/Cellar/tree-sitter/0.20.7: 18 files, 12.8MB
==> Running `brew cleanup tree-sitter`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
Warning: Calling plist_options is deprecated! Use service.require_root instead.
Please report this issue to the yqrashawn/goku tap (not Homebrew/brew or Homebrew/homebrew-core), or even better, submit a PR to fix it:
  /opt/homebrew/Library/Taps/yqrashawn/homebrew-goku/Formula/goku.rb:16

Warning: Calling plist_options is deprecated! Use service.require_root instead.
Please report this issue to the dimentium/autoraise tap (not Homebrew/brew or Homebrew/homebrew-core), or even better, submit a PR to fix it:
  /opt/homebrew/Library/Taps/dimentium/homebrew-autoraise/autoraise.rb:29

❯ brew tap d12frosted/emacs-plus
❯ brew uninstall emacs-plus@29; brew install emacs-plus@29
==> Fetching d12frosted/emacs-plus/emacs-plus@29
==> Cloning https://github.com/emacs-mirror/emacs.git
Cloning into '/Users/tzlatanov/Library/Caches/Homebrew/emacs-plus@29--git'...
==> Checking out branch emacs-29
Already on 'emacs-29'
Your branch is up to date with 'origin/emacs-29'.
==> Installing emacs-plus@29 from d12frosted/emacs-plus
==> Patching
==> Applying fix-window-role.patch
patching file 'src/nsterm.m'
==> Applying system-appearance.patch
patching file 'src/frame.h'
patching file 'src/nsfns.m'
patching file 'src/nsterm.m'
==> Applying round-undecorated-frame.patch
patching file 'src/frame.c'
patching file 'src/frame.h'
patching file 'src/nsfns.m'
patching file 'src/nsterm.h'
patching file 'src/nsterm.m'
==> ./autogen.sh
==> ./configure --enable-locallisppath=/opt/homebrew/share/emacs/site-lisp --infodir=/opt/homebrew/Cellar/emacs-plus@29/29.0.60/share/info/emacs --prefix=/opt/homebrew/Cellar/emacs-plus@29/29.0.60 --with-xml2 --with-gnutls --without-compre
==> gmake
Last 15 lines from /Users/tzlatanov/Library/Logs/Homebrew/emacs-plus@29/03.gmake:
gmake[2]: Leaving directory '/private/tmp/emacs-plusA29-20230219-25164-1erkc8e/lisp'
gmake[1]: *** [Makefile:533: lisp] Error 2
gmake[1]: Leaving directory '/private/tmp/emacs-plusA29-20230219-25164-1erkc8e'
gmake[1]: Entering directory '/private/tmp/emacs-plusA29-20230219-25164-1erkc8e'
***
*** "make all" failed with exit status 2.
***
*** You could try to:
*** - run "make bootstrap", which might fix the problem
*** - run "make V=1", which displays the full commands invoked by make,
***   to further investigate the problem
***
gmake[1]: *** [Makefile:414: advice-on-failure] Error 2
gmake[1]: Leaving directory '/private/tmp/emacs-plusA29-20230219-25164-1erkc8e'
gmake: *** [Makefile:370: all] Error 2

If reporting this issue please do so at (not Homebrew/brew or Homebrew/homebrew-core):
  https://github.com/d12frosted/homebrew-emacs-plus/issues

These open issues may also help:
Could not install emacs-plus@29 https://github.com/d12frosted/homebrew-emacs-plus/issues/544
Could not install emacs-plus@29 with any options https://github.com/d12frosted/homebrew-emacs-plus/issues/551

[tzz]

Here are all the logs config.log 03.gmake.log 02.configure.log 01.autogen.sh.log 00.options.out.log

[d12frosted]

I the the errors related to portable dump feature. Not sure why you get them. Will investigate. Thanks for sharing all the logs.

[d12frosted]

🤔 There is a thread related to this issue. Seems like some cache related stuff.

Theoretically, you can open [email protected] file and add system "gmake", "-j4 bootstrap" before the following line:

https://github.com/d12frosted/homebrew-emacs-plus/blob/496c1a358122d17b141efbc9d370b12220aeb3d5/Formula/emacs-plus%4029.rb#L181

And then you can simply run ./build 29 in the root of the Emacs+ repository.

[tzz]

system "gmake", "-j4 bootstrap"

better as system "gmake", "-j4", "bootstrap" maybe?

...so I tried that, in a local checkout of this repo, with ./build 29. Unfortunately I get the same error. In ~/Library/Logs/Homebrew/emacs-plus-local/03.gmake

2023-02-19 12:50:37 +0000

gmake
-j4
bootstrap

...
rm -f bootstrap-emacs.pdmp
./temacs --batch  -l loadup --temacs=pbootstrap \
        --bin-dest /opt/homebrew/Cellar/emacs-plus-local/29.0.60/bin/ --eln-dest /opt/homebrew/Cellar/emacs-plus-local/29.0.60/lib/emacs/29.0.60/
Loading loadup.el (source)...
Dump mode: pbootstrap
...
Dumping under the name bootstrap-emacs.pdmp
Dumping fingerprint: 3113bd374d0e934ce004145e6fd6317c6dcc5371eec4572ba4efc6bbe96b10cf
Dump complete
Byte counts: header=100 hot=15362068 discardable=148832 cold=10772144
Reloc counts: hot=1072587 discardable=5335
...
rm -f emacs && cp -f temacs emacs
LC_ALL=C ./temacs -batch  -l loadup --temacs=pdump \
        --bin-dest /opt/homebrew/Cellar/emacs-plus-local/29.0.60/bin/ --eln-dest /opt/homebrew/Cellar/emacs-plus-local/29.0.60/lib/emacs/29.0.60/
Loading loadup.el (source)...
Dump mode: pdump
...
Dumping under the name emacs.pdmp
Dumping fingerprint: 3113bd374d0e934ce004145e6fd6317c6dcc5371eec4572ba4efc6bbe96b10cf
Dump complete
Byte counts: header=100 hot=11495932 discardable=148832 cold=5164600
Reloc counts: hot=660884 discardable=5310
Adding name emacs-29.0.60.1
Adding name emacs-29.0.60.1.pdmp
cp -f emacs.pdmp bootstrap-emacs.pdmp
...
rm -f bootstrap-emacs.pdmp
./temacs --batch  -l loadup --temacs=pbootstrap \
        --bin-dest /opt/homebrew/Cellar/emacs-plus-local/29.0.60/bin/ --eln-dest /opt/homebrew/Cellar/emacs-plus-local/29.0.60/lib/emacs/29.0.60/
Loading loadup.el (source)...
Dump mode: pbootstrap
...
Dumping under the name bootstrap-emacs.pdmp
Dumping fingerprint: 878b8d8c636cb2d7784c15bfabc00e04957c2ee4332c393634cd7500a07db3f8
Dump complete
Byte counts: header=100 hot=12085660 discardable=148832 cold=5437912
Reloc counts: hot=684371 discardable=5310
...
desired fingerprint: 878b8d8c636cb2d7784c15bfabc00e04957c2ee4332c393634cd7500a07db3f8
found fingerprint: 3113bd374d0e934ce004145e6fd6317c6dcc5371eec4572ba4efc6bbe96b10cf
emacs: could not load dump file "/private/tmp/emacs-plus-local-20230219-6733-19wvu0z/src/emacs.pdmp": not built for this Emacs executable

So it looks like it finds the 311... fingerprint, but wants the last one 878.... I don't see any place where the file with the 878... fingerprint dumped under the name bootstrap-emacs.pdmp is copied to emacs.pdmp. So that may be the issue. I'm attaching the full logs analyzed above.

03.gmake.log

[tzz]

Hi. I can confirm I did cp src/bootstrap-emacs.pdmp src/emacs.pdmp after the failure and the build worked just fine, so the fingerprint matched from that point on.

Is there any chance emacs-plus could do that as well with a patch to the makefile? It would let me use the formula instead of hand-rolling the build on my machine. I'm happy to keep debugging but am quite puzzled why I'm the only one running into this problem.

[tzz]

Update: I didn't hear back so I completely wiped my Homebrew install and that seems to have fixed it for me.

I don't know the root cause but am OK with closing this issue if it affects no one else.

[danp]

I just ran into this as well trying to update an install of 29 from December. rm -rf ~/Library/Caches/Homebrew/emacs-plus@29--git seems to have sorted it for me. So my sequence of commands ended up looking something like:

brew uninstall emacs-plus@29
brew install emacs-plus@29 --with-xwidgets --with-native-comp
# got error
rm -rf ~/Library/Caches/Homebrew/emacs-plus@29--git/
brew cleanup
brew install emacs-plus@29 --with-xwidgets --with-native-comp
# worked!