[Support]: Use of `:wait` over `(elpaca-wait)`?

[rytswd]

Confirmation

• [X] I have checked the documentation (README, Wiki, docstrings, etc)
• [ ] I am checking these without reading them.
• [X] I have searched previous issues to see if my question is a duplicate.

Elpaca Version

Elpaca 49da183 grafted, HEAD -> master, origin/master, origin/HEAD
installer:      0.7
emacs-version:  GNU Emacs 29.3 (build 1, aarch64-apple-darwin23.4.0, NS appkit-2487.50 Version 14.4.1 (Build 23E224))
git --version:  git version 2.44.0

Operating System

macOS Sonoma 14.4.1

Description

First of all, thanks very much for making Elpaca, it has been a great pleasure interacting with it for over a year now! 🥰

As titled, I just noticed the recent addition of :wait, and other recent changes caused my config to get stuck at launch. It was because I had something like following (a quick disclaimer, I'm not too proficient with elisp, and code may be a mess):

(when (not after-init-time)
  (elpaca nil
    (message "Processing key binding packages for use-package use cases")
    (require 'general)
    (require 'pretty-hydra)
    (require 'major-mode-hydra)
    )
  (elpaca-wait))

This code no longer works with the current version of Elpaca, giving me the error about nil as the ORDER arg for (elpaca ORDER &rest BODY) does not accept nil. I can work out with :wait keyword for those relevant packages that introduces some extra keywords to use-package, but I wanted to see if that's the recommended way forward.

I'm especially curious about the following points:

1. Should I not use (elpaca-wait) anymore?
2. Why is this :wait keyword used within :ensure? I may be missing something, but :demand seems to make more sense as that would be needed anyhow?
3. I can see (elpaca nil ...) semantics being removed with 24a178e. How can I add extra logic instead, just any package name?
4. Would :ensure (:wait t) work with default use-package had I stopped using Elpaca for any reason?

Thanks for the support in advance!

[progfolio]

First of all, thanks very much for making Elpaca, it has been a great pleasure interacting with it for over a year now! 🥰

Thank you very much for giving it a try and sticking with it!

As titled, I just noticed the recent addition of :wait, and other recent changes caused my config to get stuck at launch. It was because I had something like following (a quick disclaimer, I'm not too proficient with elisp, and code may be a mess):

(when (not after-init-time)
  (elpaca nil
    (message "Processing key binding packages for use-package use cases")
    (require 'general)
    (require 'pretty-hydra)
    (require 'major-mode-hydra)
    )
  (elpaca-wait))

This code no longer works with the current version of Elpaca, giving me the error about nil as the ORDER arg for (elpaca ORDER &rest BODY) does not accept nil. I can work out with :wait keyword for those relevant packages that introduces some extra keywords to use-package, but I wanted to see if that's the recommended way forward.

I'm especially curious about the following points:

1. Should I not use (elpaca-wait) anymore?

You can still use it at the top-level, but I recommend using the recipe keyword instead. The rationale is that if you move the declaration, the :wait recipe keyword moves with it. Whereas, with the top-level form, one would have to move that as well. It's functionally equivalent, but more self-contained. e.g.

(elpaca (example :wait t))

;;versus

(elpaca example)
(elpaca-wait)

2. Why is this :wait keyword used within :ensure?

I decided to change Elpaca's use-package integration so that it leverages the :ensure keyword instead of adding a new keyword (formerly the :elpaca keyword). I'm hoping to convince other package manager maintainers to do the same. That should make it easier for end users when recipes are mostly equivalent between package managers.

:demand seems to make more sense as that would be needed anyhow?

:ensure is use-package's keyword for "ensure this is installed". It normally only accepts a boolean value, but I've made it so it can accept a recipe as well. You can think of it is "ensure this package is installed, using this recipe". The upside is that many use-package declarations in the wild should just work the same as they would with package.el. For example:

(use-package example :ensure t)

That should work with Elpaca or package.el.

:demand is concerned with loading a package. That's a separate, later step than installing/activating a package. It's valid, and often desirable to defer loading of a package. It speeds up the init time of Emacs and reduces its memory footprint. Many of use-package's other keywords implicitly defer loading (off the top of my head, :commands, :bind, :hook, :mode, etc). There's a good section on lazy loading in use-package's Info manual that's worth a read.

3. I can see (elpaca nil ...) semantics being removed with 24a178e. How can I add extra logic instead, just any package name?

That pattern was removed because it wasn't semantically correct. There was no way to tell multiple invocations of it apart because they all used nil as their ID.

If something needs to happen after a package is loaded, you can use use-package's :config command or, sans use-package, with-eval-after-load or in the BODY of a elpaca declaration. e.g.

(use-package example :config (message "example has been loaded"))

;; or

(with-eval-after-load 'example (message "example has been loaded"))

;; or

(elpaca example (require 'example) (message "example has been loaded"))

Or, to use your example:

;; Assuming `elpaca-use-package-mode' active and `use-package-always-ensure' non-nil...
(unless after-init-time
  (message "Processing key binding packages for use-package use cases"))
(use-package general)
(use-package pretty-hydra)
(use-package major-mode-hydra :ensure (:wait t))

4. Would :ensure (:wait t) work with default use-package had I stopped using Elpaca?

Not currently. It would warn with the following:

⛔ Error (use-package): Failed to parse package example: use-package: :ensure wants an optional package name (an unquoted symbol name), or (<symbol> :pin <string>)

You would have to replace (:wait t) with t or one of the values mentioned in the above warning. Most other package managers have no equivalent concept to elpaca-wait because they do not install packages asynchronously.

Does that help?

[rytswd]

@progfolio Sorry for my delayed response, and thanks very much for the detailed information here, they are all really helpful!

The rationale is that if you move the declaration, the :wait recipe keyword moves with it.

This was actually the biggest gain that I only noticed after migrating to use :wait keyword. I have key binding setup using general.el, pretty-hydra, and major-mode-hydra, all of which add use-package keyword, and I needed to ensure that was made available before other packages are loaded up. It should theoretically mean that I don't have to think about the load order, and it would be handled gracefully by Elpaca if I'm not mistaken?

As to the :demand vs :ensure, the reason I thought :demand t is necessary is because of these use-package related packages. I thought installing won't be sufficient, but it looks like as long as :ensure is in place, those packages would be loaded up anyways as they are being used during start-up by other use-package definitions. Thanks for the pointer!

I appreciate all the details, clarifying all the questions I had! I just got one last question, though. In the example above, you touched on how I could use after-init-time hook:

(use-package general)
(use-package pretty-hydra)
(use-package major-mode-hydra :ensure (:wait t))

Do you mean that, if I want to have all of those three packages to be "waited" before other use-package, I assume I would want :ensure (:wait t) in all packages?

[progfolio]

@progfolio Sorry for my delayed response, and thanks very much for the detailed information here, they are all really helpful!

Not a problem. Glad the information helped.

It should theoretically mean that I don't have to think about the load order, and it would be handled gracefully by Elpaca if I'm not mistaken?

Wherever you put a :wait t, all of the queued packages up to that point will be processed before moving on.

I just got one last question, though. In the example above, you touched on how I could use after-init-time hook:

The after-init-time in my example is the variable. It was a translation of the example you gave here:

(when (not after-init-time)
  (elpaca nil
    (message "Processing key binding packages for use-package use cases")
    (require 'general)
    (require 'pretty-hydra)
    (require 'major-mode-hydra)
    )
  (elpaca-wait))

(when (not... is equivalent to (unless ...

(use-package general)
(use-package pretty-hydra)
(use-package major-mode-hydra :ensure (:wait t))

Do you mean that, if I want to have all of those three packages to be "waited" before other use-package, I assume I would want :ensure (:wait t) in all packages?

You could do it that way, but that would slow things down a bit. It would basically force those three packages to be processed synchronously. If those three packages can be processed in parallel, there's no harm in leaving it how you have it now with :wait t in the last of the three. It's basically flexible enough where all of these will work:

The slowest way (wouldn't recommend in this case):

  (use-package general :ensure (:wait t)) ;; wait until done
  (use-package pretty-hydra :ensure (:wait t)) ;;wait until done
  (use-package major-mode-hydra :ensure (:wait t)) ;;wait until done

This will wait until all three are processed, but they'll be processed in parallel (faster):

(use-package general)
(use-package pretty-hydra)
(use-package major-mode-hydra :ensure (:wait t))

So will this:

(progn
  (use-package general)
  (use-package pretty-hydra)
  (use-package major-mode-hydra)
  (elpaca-wait))

[rytswd]

@progfolio Ah thanks for the details and corrections, they all make sense! (I mixed up of after-init-time and after-init-hook 🫠)

I'm unsure if this addition of :wait is necessarily cleaner, though. When there are multiple packages that need to be loaded and waited in order to add extra use-package keywords, I have to ensure the order of use-package is kept, as in:

(use-package general)
(use-package pretty-hydra)
(use-package major-mode-hydra :ensure (:wait t))

This would work, but when I decide to move things around:

(use-package pretty-hydra)
(use-package major-mode-hydra :ensure (:wait t))
(use-package general)

This won't work for general.el integration if I understand it correctly.

Because of the async nature of Elpaca, it would be the best IMO if I didn't have to think about the actual order of use-package definitions, and simply mark the package saying this one needs to be loaded earlier than others -- and in that sense I thought :wait could be applied to all those that need to be loaded before other packages are loaded.

For my own setup, I think I will probably stick with (elpaca-wait). These are completely personal, but I thought I'd share my thought process so that you may be able to understand where I'm coming from:

• The use of :wait would need to be learned as it's specific for Elpaca (and could be foreign for any new to my config code)
• The wait is not about one package but all packages that are added to the queue up to that point, which needs its own explanation (for future myself)
• I would one day forget its meaning and try to remove it, or move other definitions out of order 🫠
• The use of (elpaca-wait) can be explicitly documented / commented, and to me it is a simpler solution

[progfolio]

@progfolio Ah thanks for the details and corrections, they all make sense! (I mixed up of after-init-time and after-init-hook 🫠)

You're welcome!

Because of the async nature of Elpaca, it would be the best IMO if I didn't have to think about the actual order of use-package definitions

I had a sense of déjà vu reading that. :) I felt the same way 5 years ago about use-package declarations:

https://github.com/jwiegley/use-package/issues/705#issuecomment-510237726

In the case of general, and similar packages which modify use-package or add something you'd like to use at the top-level of your init file, the order of the declaration matters regardless of what package manager is being used.

I'm unsure if this addition of :wait is necessarily cleaner, though. For my own setup, I think I will probably stick with (elpaca-wait).

That works, too. You can wrap the forms in a progn as in my previous comment, or you could wrap them in the elpaca-queue macro (though I've been considering getting rid of that, so no promises on that sticking around forever). As you point out, I think it's best to stick with whatever you'll find most maintainable.

[rytswd]

Just to close off the thread and leave a bit of reference to others who may stumble on this later, I ended up with the following setup:

• Do not use :wait keyword with each package
• Define all the packages that need to be loaded up early (such as general, pretty-hydra, etc. which touches use-package) in a dedicated file
• Configure to load that file early in init.el, and then run (elpaca-wait) to handle the queue at that point

Something like below:

;; Load key bindings early
(load (expand-file-name "key-binding-core.el" user-emacs-directory)) 

;; Ensure to handle the use-package up to this point, so that I can get the extra
;; keywords for the rest of packages.
(when (not after-init-time)
    (elpaca-wait))

;; ... handle more packages ...

(I kept the explicit check for init, so that I could do something like (load (expand-file-name "init.el" user-emacs-directory)) for reloading the entire config without doing unnecessary wait -- though as I'm using use-package extensively, I have a better chance just starting up a new Emacs instance... 🫠)

Thanks very much for all the help in the thread, though! I was thinking of doing elpaca-queue but decided against as it sounded a bit odd how I had to call "queue" which sounds heavily reliant on implementation details.