"ensure" fails if the package version has changed on the remote repository

[rdiez]

I am not sure whether use-package is the right place for this bug report.

I wanted to install the "which-key" package, so I added this to my config:

(use-package which-key :ensure t :config (which-key-mode))

I did not want to restart Emacs, so I ran the code above manually. I got the following error:

Error (use-package): Failed to install which-key: https://melpa.org/packages/which-key-20190315.1248.el: Not found Error (use-package): Cannot load which-key

I did some digging, and it turns out that I had not updated the package list in a few weeks. Apparently, MELPA does not serve old package versions.

This is all rather unfortunate. Imagine you run "sudo apt install some-package" on Debian, and that fails if the package version on the remote repository has been updated in the mean time. You would then complain that "apt install" is not actually reliable. Installing a package should be reliable, even if a newer version is available. Whether it installs the older or the newer version is debatable.

[ThibautVerron]

Does it simply mean that we should add a line to update the package list at the beginning of our init files?

[ThibautVerron]

Is it a duplicate of https://github.com/jwiegley/use-package/issues/256 ?

[rdiez]

I am not sure that this bug is a duplicate #256 . That bug talks about bootstrapping, installing new packages, and so on. It is quite a long bug.

My case is rather simpler: the current package list knows that the given package exists, but it fails to download it because the exact URL is no longer available. It turns out that the package is still on the repository, with the same package name too, but only its version has increased in the mean time, and the older package version no longer exists under that URL on the server.

I do not know how this all works. It may be MELPA's fault. I am wondering whether "use-package" could download the metadata for that package, and not just try to download the exact URL with the exact older package version it knows about. Alternatively, if the package is still there, has been updated in the mean time, and the old URL is no longer available, "use-package" could generate a better error message like this:

A newer package version has been uploaded to the repository, and the old version is no longer available. Please update your package list and try again.

Alternatively, "use-package" could download and update the metadata just for the requested package, if that is possible. Or even ask: 1) Download only the newer package version, or 2) Update the complete package list before downloading the package. But that may not be doable from within "use-package".

[ThibautVerron]

I think that it is the same, see for example this comment: https://github.com/jwiegley/use-package/issues/256#issuecomment-170331552 :

Installation will fail if an old version of the package appears in package.el's cache but the version on the server is newer.

Isn't it exactly what you are reporting?

Use-package itself does not handle package installation, metadata, etc, instead it defers to a package manager (by default package.el).

1. Download only the newer package version, or 2) Update the complete package list before downloading the package. But that may not be doable from within "use-package".

Wouldn't it be the exact same result?

[rdiez]

Yes, that particular comment matches what this bug is about.

I am not sure that doing a package-refresh-contents once per session is a good idea. That would probably make Emacs pause on every start-up. A better error message would go a long way to mitigate this issue.

[thinkhuman]

I had the same error; M-x package-refresh-contents (and an Emacs restart) resolved the problem for me.

[rdiez]

Of course, manually refreshing and restarting will fix the problem. But you have to know what the problem is, and it is not immediately obvious that use-package has this limitation. The first thing you suspect when you see the "not found" error is that you got the package name wrong, or that the package no longer exists in the repository.

[conao3]

https://github.com/conao3/leaf.el/blob/a4fd520f5c31f54e0797155866e0b35df277664e/leaf.el#L1081-L1087 leaf try to solve this issue. If failed to package-install, try to package-refresh-contents and then try package-install. If second package-install failed, finally leaf give up and raise error.

[skangas]

https://github.com/conao3/leaf.el/blob/a4fd520f5c31f54e0797155866e0b35df277664e/leaf.el#L1081-L1087 leaf try to solve this issue. If failed to package-install, try to package-refresh-contents and then try package-install. If second package-install failed, finally leaf give up and raise error.

I think this is what use-package should do in this case too.

[michaelmhoffman]

Would suggest addition of a new function to call package-refresh-packages and set up pinning correctly:

(defun use-package-refresh-packages ()
  (package-refresh-contents)
  (when (assoc package (bound-and-true-p
                        package-pinned-packages))
    (package-read-all-archive-contents)))

And then edits to use-package-ensure-elpa to call this function on the first error when calling package-install or when the package can't be found inpackage-archive-contents.

(defun use-package-ensure-elpa (name args _state &optional _no-refresh)
  (dolist (ensure args)
    (let ((package
           (or (and (eq ensure t) (use-package-as-symbol name))
               ensure)))
      (when package
        (require 'package)
        (when (consp package)
          (use-package-pin-package (car package) (cdr package))
          (setq package (car package)))
        (unless (package-installed-p package)
          (condition-case-unless-debug err
              (progn
                (when (assoc package (bound-and-true-p
                                      package-pinned-packages))
                  (package-read-all-archive-contents))
                (if (assoc package package-archive-contents)
                    (condition-case _err
                        (package-install package)
                      (error
                       (use-package-refresh-packages)
                       (package-install package)))
                  (use-package-refresh-packages)
                  (package-install package))
                t)
            (error
             (display-warning 'use-package
                              (format "Failed to install %s: %s"
                                      name (error-message-string err))
                              :error))))))))