`org-mode` is broken after restarting emacs on latest develop when emacs < 29

[nixmaniack]

Description :octocat:

org-mode is broken. Opening an org file doesn't load the org-mode.

I tried removing all org packages, restart emacs to let it install all the packages and then open an org file which works for that session. If you quit and restart it throws (invalid-function org-assert-version) error.

Reproduction guide :beetle:

• Delete org-9.6.1 and org-contrib-0.4.1 from ~/.emacs.d/elpa/28.2/develop.
• Start emacs.
• Let it install packages.
• Open any org file, this loads org-mode file for the session.
• Restart emacs.
• Open any org file, it throws error.

Observed behaviour: :eyes: :broken_heart: Opening an org file throws following error.

File mode specification error: (invalid-function org-assert-version)

Expected behaviour: :heart: :smile: Opens file and applies org-mode and related layer configuration.

System Info :computer:

• OS: gnu/linux
• Emacs: 28.2.50
• Spacemacs: 0.999.0
• Spacemacs branch: develop (rev. d08822c61)
• Graphic display: t
• Running in daemon: nil
• Distribution: spacemacs
• Editing style: hybrid
• Completion: ivy
• Layers:

(auto-completion emacs-lisp git ivy lsp markdown multiple-cursors org
                 (shell :variables shell-default-height 30 shell-default-position 'bottom shell-default-shell 'vterm)
                 syntax-checking version-control treemacs themes-megapack)

• System configuration features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

[lebensterben]

cc @smile13241324

[lebensterben]

Confirmed. At first my emacs was working fine. But after deleting the two packages mentioned above, Emacs won't start properly. (spacemac-buffer won't show up since it requires org with my settings)

[lebensterben]

Inspired by https://irreal.org/blog/?p=10999

To fix it:

1. cd ~/.emacs.d; rm -rf elpa/28.2/develop/org-9*

2. start emacs with emacs -Q and evaluate (with C-j)

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir
      (expand-file-name "develop"
                        (expand-file-name emacs-version
                                          (expand-file-name "elpa"
                                                            user-emacs-directory))))
(package-refresh-contents)
(package-install 'org)

3. Restart Emacs normally.

[nixmaniack]

@lebensterben I followed your workaround but looks it didn't work for me. I did rm -rf the org packages but the last line to install org throws this message.

"‘org’ is already installed"

I just followed your exact instruction and haven't dug into the blog post. I will check what's going on later. One thing I noticed is that, emacs-version is reported as 28.2.50 which makes the path elpa/28.2.50/develop but all the other packages spacemacs has installed are in elpa/28.2/develop.

[lebensterben]

make sure you removed all org-9.6* in your elpa directory.

[smile13241324]

Strange, this seems only to happen on emacs 28, not on emacs 29 and it seems to be independent from what is entered into the spacemacs buffer, setting it to show no content at all (no lists no agenda) has the same effect.

When I am trying to require org in my dotfile I am seeing the following:

Debugger entered--Lisp error: (invalid-function org-assert-version)
  org-assert-version()
  byte-code("\300\301!\210\302 \210\300\303!\210\300\304!\207" [require org-macs org-assert-version cl-lib oc] 2)
  #<subr require>(org-keys)
  apply(#<subr require> org-keys)
  require(org-keys)
  eval-buffer(#<buffer  *load*> nil "/home/smile13241324/.emacs.d/elpa/28.2/develop/org..." nil t)  ; Reading at buffer position 3557
  load-with-code-conversion("/home/smile13241324/.emacs.d/elpa/28.2/develop/org..." "/home/smile13241324/.emacs.d/elpa/28.2/develop/org..." nil t)
  #<subr require>(org)
  apply(#<subr require> org)
  require(org)
  (progn (require 'org))
  eval((progn (require 'org)) t)
  elisp--eval-last-sexp(nil)
  #f(compiled-function (eval-last-sexp-arg-internal) "Evaluate sexp before point; print value in the echo area.\nInteractively, with a non `-' prefix argument, print output into\ncurrent buffer.\n\nThis commands handles `defvar', `defcustom' and `defface' the\nsame way that `eval-defun' does.  See the doc string of that\nfunction for details.\n\nNormally, this function truncates long output according to the\nvalue of the variables `eval-expression-print-length' and\n`eval-expression-print-level'.  With a prefix argument of zero,\nhowever, there is no such truncation.\nInteger values are printed in several formats (decimal, octal,\nand hexadecimal).  When the prefix argument is -1 or the value\ndoesn't exceed `eval-expression-print-maximum-character', an\ninteger value is also printed as a character of that codepoint.\n\nIf `eval-expression-debug-on-error' is non-nil, which is the default,\nthis command arranges for all errors to enter the debugger." (interactive "P") #<bytecode -0x45c973c4e971b99>)(nil)
  #f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>)()
  eval-sexp-fu-flash-doit-simple(#f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>) #f(compiled-function (&rest args2) #<bytecode -0x1133d5a2cfbd057f>) #f(compiled-function (&rest args2) #<bytecode -0x112bf59b0182057f>))
  eval-sexp-fu-flash-doit(#f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>) #f(compiled-function (&rest args2) #<bytecode -0x1133d5a2cfbd057f>) #f(compiled-function (&rest args2) #<bytecode -0x112bf59b0182057f>))
  esf-flash-doit(#f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>) #f(compiled-function (&rest args2) #<bytecode -0x1133d5a2cfbd057f>) #f(compiled-function (&rest args2) #<bytecode -0x112bf59b0182057f>) #f(compiled-function (&rest args2) #<bytecode 0x153e757edc0c5a87>))
  ad-Advice-eval-last-sexp(#f(compiled-function (eval-last-sexp-arg-internal) "Evaluate sexp before point; print value in the echo area.\nInteractively, with a non `-' prefix argument, print output into\ncurrent buffer.\n\nThis commands handles `defvar', `defcustom' and `defface' the\nsame way that `eval-defun' does.  See the doc string of that\nfunction for details.\n\nNormally, this function truncates long output according to the\nvalue of the variables `eval-expression-print-length' and\n`eval-expression-print-level'.  With a prefix argument of zero,\nhowever, there is no such truncation.\nInteger values are printed in several formats (decimal, octal,\nand hexadecimal).  When the prefix argument is -1 or the value\ndoesn't exceed `eval-expression-print-maximum-character', an\ninteger value is also printed as a character of that codepoint.\n\nIf `eval-expression-debug-on-error' is non-nil, which is the default,\nthis command arranges for all errors to enter the debugger." (interactive "P") #<bytecode -0x45c973c4e971b99>) nil)
  apply(ad-Advice-eval-last-sexp #f(compiled-function (eval-last-sexp-arg-internal) "Evaluate sexp before point; print value in the echo area.\nInteractively, with a non `-' prefix argument, print output into\ncurrent buffer.\n\nThis commands handles `defvar', `defcustom' and `defface' the\nsame way that `eval-defun' does.  See the doc string of that\nfunction for details.\n\nNormally, this function truncates long output according to the\nvalue of the variables `eval-expression-print-length' and\n`eval-expression-print-level'.  With a prefix argument of zero,\nhowever, there is no such truncation.\nInteger values are printed in several formats (decimal, octal,\nand hexadecimal).  When the prefix argument is -1 or the value\ndoesn't exceed `eval-expression-print-maximum-character', an\ninteger value is also printed as a character of that codepoint.\n\nIf `eval-expression-debug-on-error' is non-nil, which is the default,\nthis command arranges for all errors to enter the debugger." (interactive "P") #<bytecode -0x45c973c4e971b99>) nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)

I think we need more investigation into this first.

[nixmaniack]

I had to explicitly set the package-user-dir to a value that normal spacemacs sets up and I was able to install org with the instructions provided. For now, the workaround seems to work. Thanks @lebensterben!

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir "/home/localuser/.emacs.d/elpa/28.2/develop/")
(package-refresh-contents)
(package-install 'org)

[hny-gd]

For me running

cd ~/.emacs.d; rm -rf elpa/develop/org-9* (note develop instead of 28.2 as @lebensterben wrote above)

and then starting Emacs was enough.

Spacemacs then reinstalled org and everything was fine.

[emacs18]

There has been many problems like this over the years due to org mode. Emacs comes with org mode, but almost always folks need to install a newer version. This newer version is also needed by other packages as well. That is why I make sure that org package is installed very early on in early-init.el. I used to do this when I used package.el years ago. Since I've been using straight.el for few years, I have (straight-use-package 'org) very early on in my early-init.el file to assure that org package is installed before even getting to most spacemacs initialization code.

[RidaAyed]

I had to explicitly set the package-user-dir to a value that normal spacemacs sets up and I was able to install org with the instructions provided. For now, the workaround seems to work. Thanks @lebensterben!

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir "~/.emacs.d/elpa/28.2/develop/")
(package-refresh-contents)
(package-install 'org)

Note that in my case, alongside the existing package directory (e.g. ~/.emacs.d/elpa/develop) the upper snippet downloads and installs all packages into the package-user-dir and finally reports: ‘org’ is already installed

make sure you removed all org-9.6* in your elpa directory.

M-x org-version still reports Org mode version 9.5.5 (release_9.5.5 @ /usr/share/emacs/28.2/lisp/org/)

[lebensterben]

@RidaAyed

https://github.com/syl20bnr/spacemacs/issues/15896#issuecomment-1397716997

have you first removed org from elpa directory?

[emacs18]

I don't think org package is in melpa. You need to add https://elpa.gnu.org/packages/. Following is how spacemacs sets up packages.

(defun configuration-layer/create-elpa-repository (name output-dir)
  "Create an ELPA repository containing all packages supported by Spacemacs."
  (configuration-layer/make-all-packages 'no-discover)
  (let (package-archive-contents
        (package-archives '(("melpa" . "https://melpa.org/packages/")
                            ("gnu"   . "https://elpa.gnu.org/packages/")
                            ("nongnu" . "https://elpa.nongnu.org/nongnu/"))))

[lebensterben]

if one followed my instructions melpa is added.

[notestaff]

FWIW, removing the byte-compiled *.elc files in the installed elpa org directory seems to fix the org-assert-function issue for me (but I use standard emacs not spacemacs).

[emacs18]

Corrupt byte compiled files could result if you installed org package after already having loaded built-in org.el. This is why one must make sure that org.el is not loaded when you install org package! This is why I have code in early-init.el (rather than init.el) to not only setup package archives, but also to install org packages. Because this is done so early on during emacs start-up, there is no possibility of org.el being loaded when org package is installed.

I think this is the most common problem with org mode the folks have had over the years.

[lebensterben]

@emacs18 will there be any impact on loading time?

[emacs18]

I don't think so, but I am not sure. Even if startup time is longer, I could tolerate it if that prevents hard to debug errors.

[tko]

I think this might get fixed by deleting all of elpa/ -- no guarantees though. (Two computers, on one I deleted everything for unrelated reasons and it works ok, on the other I've just kept upgrading things and hit this same error. Was planning to delete elpa when I have the time.)

[sir4ur0n]

@tko I just tested and it doesn't solve the problem for me 😞

[RidaAyed]

Inspired by https://irreal.org/blog/?p=10999

To fix it:

1. `cd ~/.emacs.d; rm -rf elpa/28.2/develop/org-9*`

2. start emacs with `emacs -Q` and evaluate (with `C-j`)

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir
      (expand-file-name "develop"
                        (expand-file-name emacs-version
                                          (expand-file-name "elpa"
                                                            user-emacs-directory))))
(package-refresh-contents)
(package-install 'org)

3. Restart Emacs normally.

@lebensterben

Running your snippet returns "‘org’ is already installed"

Yes, even deleted the complete elpa folder. Tried even deleting /usr/share/emacs/28.2/lisp/org/ to no avail. After reinstalling emacs yay emacs > reinstall I'm back to "‘org’ is already installed"

@lebensterben ..and of course: Thank you very much for your time :pray:

[lebensterben]

@RidaAyed

what's your emacs version

[RidaAyed]

@lebensterben

System Info :computer:

• OS: gnu/linux
• Emacs: 28.2
• Spacemacs: 0.999.0
• Spacemacs branch: develop (rev. 8645f21cb)
• Graphic display: t
• Running in daemon: nil
• Distribution: spacemacs
• Editing style: vim
• Completion: ivy

[emacs18]

@RidaAyed

Can you explain how hour setup can install org package at all?

You set package-archives to include only melpa. As far as I can tell, melpa does not provide org package! Thus I don't see how you could possibly be installing org package when package-archives does not include https://elpa.gnu.org/packages/ as well.

[emacs18]

Sorry. I just noticed that you are using add-to-list. So you are probably relying on default value of package-archive to already have elpa.gnu.org.

[RidaAyed]

@emacs18 I was referring to to the snippet from @lebensterben

Otherwise I'm relying on the standard spacemacs layers

dotspacemacs-configuration-layers
'(
  (org :variables
       org-enable-hugo-support t
       org-enable-valign nil
       )
  )

[nixmaniack]

@RidaAyed Your snippet sets the value of the package-user-dir to a different value than what spacemacs computes. I had similar issues when I was debugging hence I hardcoded the value for the workaround. You can check how spacemacs computes the value here but in short it strips the patch version from the version string. So even though the workaround downloads the fixed org version, it's never used by Spacemacs.

For clarity, on my machine, Spacemacs computes package-user-dir as /home/localuser/.emacs.d/elpa/28.2/develop/, and your snippet computes it as /home/localuser/.emacs.d/elpa/28.2.50/develop.

[lebensterben]

My snippet would work if one uses the default value of dotspacemacs-elpa-subdirectory and a stable release of Emacs.

otherwise, one should use https://github.com/syl20bnr/spacemacs/blob/a529b0bfa8fb04c368e6d57059929806b9722e2f/core/core-configuration-layer.el#L440-L442 to calculate the correct path and hardcode it as @nixmaniack pointed out.

[vitaminace33]

As of today, I did a clean install. After that I had to delete org-9.6.1 and let Emacs reinstall it.

[RidaAyed]

I've changed dotspacemacs-elpa-subdirectory from nil to emacs-version

Describe variable package-user-dir returns:

  Its value is "/home/localuser/.emacs.d/elpa/28.2/develop/"
  Original value was "~/.emacs.d/elpa"

However deleting the complete elpa folder followed by the snippet from @lebensterben doesn't help. The snippet returns org mode is already installed

Since org-9.6 in fact resides in /home/localuser/.emacs.d/elpa/28.2/develop/ after installation I can't understand why Spacemacs doesn't pick it up and org-version remains 9.5.5.

@nixmaniack I couldn't reproduce how you got /home/localuser/.emacs.d/elpa/28.2.50/develop

[nixmaniack]

So, using the first snippet to compute the value of package-user-dir makes the workaround useless because Spacemacs never looks in that directory (at least in my case since I'm using emacs version 28.2.50).

I would suggest following:

1. Run Spacemacs normally, and check the value of package-user-dir.
2. Then, use the workaround snippet by hardcoding the value of the package-user-dir