`use-package-always-ensure` also adds built-in packages into `package-selected-packages` and will connect to archives to get built-in packages on every startup

[AlynxZhou]

I can reproduce this with following init.el:

(require 'package)
(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))
(require 'use-package)
(setq use-package-always-ensure t)
(use-package recentf
  :defer 1
  :config
  (recentf-mode 1))

If you start Emacs with it, you can see following messages in *Messages* buffer every startup (ignore my ELPA mirror):

Importing package-keyring.gpg...done
Contacting host: mirrors.tuna.tsinghua.edu.cn:443 [2 times]
Package refresh done
‘recentf’ is already installed
Saving file /home/alynx/.emacs.d.test/init.el...
Wrote /home/alynx/.emacs.d.test/init.el

and recentf will be added into package-selected-packages.

I think use-package should not try to install built-in packages and ignore :ensure for them.

I did some test to find what makes my Emacs refresh archive contents on every startup before I found it's use-package-always-ensure, and find some interesting things:

It seems related to package-activate-all added in Emacs 27.1, after 27.1, Emacs will call package-activate-all automatically between early-init.el and init.el instead of package-initialize, because package-activate-all supports package-quickstart and will load the cache.

And if I disable this behavior by adding (setq package-enable-at-startup nil) to early-init.el, and call (package-initialize) manually in init.el, use-package works fine with use-package-always-ensure and built-in packages, but this makes packages-quickstart invalid, since package-initialize loads each package's autoload files before quickstart cache is loaded.

If I replace (package-initialize) I added in init.el with (package-activate-all), this bug happens again, and I then read the code of them to find what makes use-package behave different, I found that package-initialize contains a line (setq package--initialized t) before it calls (package-activate-all), if I replace (package-initialize) with (setq package--initialized t)(package-activate-all), use-package also works fine and quickstart is used.

So the simplest way to resolve this problem is ignore :ensure for built-in packages of Emacs, but I am also wondering why package--initialized makes use-package behave different, maybe functions to detect built-in packages depends on it? If so, is this an issue of package.el?

[jeffvalk]

I'm experiencing the same thing. If use-package-always-ensure is set, any use-package declaration for a built-in package causes Emacs to connect to the package archives on startup.

Analysis

The relevant use-package code is here: https://github.com/jwiegley/use-package/blob/a7422fb8ab1baee19adb2717b5b47b9c3812a84c/use-package-ensure.el#L165-L170

The code above seems sensible. The problem is that when package--initialized is nil, package-installed-p returns nil for built-in packages, which I find surprising. This case causes connection to the package archives.

The package.el code for package-installed-p is here:

https://github.com/emacs-mirror/emacs/blob/c5c6d5cf1c886635579142d67b743421043fe5d9/lisp/emacs-lisp/package.el#L2054-L2072

• If package--initialized is nil, the second cond test matches causing the package to be looked up in package-activated-list. For a built-in package, this will return nil.
• If package--initialized is non-nil, it falls through to the third cond clause, which returns (package-built-in-p package min-version). This gives the desired result.

Question

I would expect package-installed-p to always return t for a built-in package (when no min-version argument is specified). That is not the current behavior. Does this sound correct? If so, I believe this should be fixed in package.el.

Workarounds

In the meanwhile, a few workarounds are possible. The first involves an update to use-package. The others are user configurations.

1. Modify the test in use-package-ensure-elpa to be:

   (unless (or (package-installed-p package)
               (package-built-in-p package)))
   

2. Define a custom use-package-ensure-function with the same logic as above. (Quite heavy on the user)

3. Add (setq package--initialized t) before any built-in use-package declarations. (A vague, magical hack)

4. Add :ensure nil to use-package declarations for all built-in packages. (Cleaner, but less convenient)

[stapelberg]

Thanks for the detective work, everyone! I also ran into this, and adding :ensure nil to my use-package directives for the tramp and project packages worked for me. I agree that this is very surprising behavior…

[skangas]

I would expect package-installed-p to always return t for a built-in package (when no min-version argument is specified). That is not the current behavior. Does this sound correct? If so, I believe this should be fixed in package.el.

Which version of Emacs are you using?

On current master (Emacs 29) and Emacs 27.1, the following evaluates to t in emacs -Q here:

(progn
  (require 'package)
  (package-installed-p 'recentf))

[AlynxZhou]

I would expect package-installed-p to always return t for a built-in package (when no min-version argument is specified). That is not the current behavior. Does this sound correct? If so, I believe this should be fixed in package.el.

Which version of Emacs are you using?

On current master (Emacs 29) and Emacs 27.1, the following evaluates to t in emacs -Q here:

(progn
  (require 'package)
  (package-installed-p 'recentf))

A simple git blame tells me that this has been fixed by Emacs upstream, let's cheers and close this issue!

https://github.com/emacs-mirror/emacs/commit/50a192795ad64d2ea49274b402cb42530a5199ca

[skangas]

A simple git blame tells me that this has been fixed by Emacs upstream, let's cheers and close this issue!

emacs-mirror/emacs@50a1927

That's good news indeed, thanks for looking into it.