[Support]: Lock file usage

[aaronjensen]

Confirmation

• [x] I have checked the documentation (README, Wiki, docstrings, etc)
• [ ] I am checking these without reading them.
• [x] I have searched previous issues to see if my question is a duplicate.

Elpaca Version

Elpaca d6e99f4 HEAD -> master, origin/master, origin/HEAD installer: 0.1 emacs-version: GNU Emacs 31.0.50 (build 1, aarch64-apple-darwin24.3.0, NS appkit-2575.40 Version 15.3.2 (Build 24D81)) of 2025-03-16 git --version: git version 2.48.1

Operating System

macOS

Description

Hi, if I have a lock file specified, how do I update packages beyond their locked version? So far it seems I have to comment out the lockfile specification and then restart Emacs.

The overall process of using a lock file is not clear/obvious and the documentation doesn't seem to clarify it. I'm used to being able to specify a lock file and then any time I explicitly update, the lock file will be updated. I don't mind having to manually write it, but I'd like to be able to update packages using elpaca without having to disable the lock file entirely.

Maybe I'm totally missing something on the paradigm here, however.

Thank you.

[progfolio]

Hi, if I have a lock file specified, how do I update packages beyond their locked version? So far it seems I have to comment out the lockfile specification and then restart Emacs.

That's correct. A possible future improvement would be to introduce a global minor mode which toggles the elpaca-lock-file variable and removes any pinning keywords from packages associated with the lock file.

[kmontag]

In case it helps anyone, I've been using this setup to manage updates for locked packages. General overview:

• you can use the my/elpaca-unpin or my/elpaca-unpin-all functions to remove pinning information within the current session, and then use elpaca-fetch, elpaca-merge, etc as usual.
• when cloning/updating a pinned package, instead of the repository being left in a detached-head state, a local branch called _elpaca-current will be created which tracks the recipe's default remote branch. This fixes failures that would otherwise occur on merge operations.

AFAIK there's still no good way to "push" lockfile changes to local packages (e.g. if you update packages on one machine and then want to checkout the updated versions on a different machine). My brute-force approach is just to delete the elpaca directory and restart emacs when the packages are out of sync with the lockfile.

I'm happy to clean this up into a PR, though the creation of the _elpaca-current branch feels like a hack, and there might be a better way to deal with this more natively within elpaca. In the meantime:

(defconst my/elpaca-branch-name "_elpaca-current"
  "Name of the local branch used to track upstream changes.")

(defun my--elpaca-unpinned-recipe (e)
  "Calculate the recipe for E without lockfile influence."
  (let* ((order (elpaca<-order e))
         (elpaca-menu-functions (remove #'elpaca-menu-lock-file elpaca-menu-functions)))
    (elpaca-recipe order)))

(defun my/elpaca-remote-valid-p (e)
  "Check if E's current remote matches what would be used without the lockfile.
Returns nil if the remotes don't match or if the package isn't installed."
  (when-let* ((current-recipe (elpaca<-recipe e))
              (default-directory (elpaca<-repo-dir e))
              (recalculated-recipe (my--elpaca-unpinned-recipe e))
              (current-remote-name
               (or (car-safe (plist-get current-recipe :remotes)) elpaca-default-remote-name))
              (recalculated-remote-name
               (or (car-safe (plist-get recalculated-recipe :remotes)) elpaca-default-remote-name)))
    (let* ((remote-name
            (if (listp current-remote-name)
                (car current-remote-name)
              current-remote-name))
           (expected-remote-name
            (if (listp recalculated-remote-name)
                (car recalculated-remote-name)
              recalculated-remote-name))
           (actual-url
            (string-trim
             (elpaca-process-output "git" "config" "--get" (format "remote.%s.url" remote-name))))
           (expected-url (elpaca--repo-uri recalculated-recipe)))
      (and actual-url expected-url (string= actual-url expected-url)))))

(defun my--elpaca-configure-upstream (e)
  "If E's repo head is detached, create a remote-tracking branch from it.

The branch will be named according to `my/elpaca-branch-name'. The
remote and branch to track will be determined using `elpaca-recipe' with
the lockfile recipes excluded, and any explicitly-configured properties
included. For example, if an explicit :branch is configured in a
`use-package' declaration, it will be used as the upstream; otherwise,
the :branch from the package's MELPA (or whatever) recipe will be used;
and if neither of these is set, the repo's remote HEAD branch will be
used.

This function gets run during elpaca build/install steps after checking
out the revision. This allows e.g. `elpaca-merge' to pull in updates for
packages that normally only have a :ref configured (e.g. all packages
loaded from the lockfile), since it requires an upstream branch to run
the diff/merge."
  (let* ((default-directory (elpaca<-repo-dir e))
         (detached-p (string-match-p "HEAD detached" (elpaca-process-output "git" "branch")))
         (remote-valid (my/elpaca-remote-valid-p e))
         (id (elpaca<-id e))
         (order (elpaca<-order e))
         (branch nil))

    (when detached-p
      ;; Delete branch if it already exists.
      (elpaca-with-process-call
       ("git" "branch" "-D" my/elpaca-branch-name)
       (when success
         (elpaca--signal e (format "Deleted existing %s branch" my/elpaca-branch-name))))

      ;; Only create a new branch if the remote is valid.
      (if (not remote-valid)
          (warn
           (concat
            "Invalid remote for package %s. "
            "This probably means the package's repository URL has changed since it was installed. "
            "You may want to delete its entry from the lockfile and reinstall it.")
           id)
        (let* ((recalculated-recipe (my--elpaca-unpinned-recipe e))
               (remote
                (or (car-safe (plist-get recalculated-recipe :remotes)) elpaca-default-remote-name))
               (remote-str
                (if (listp remote)
                    (car remote)
                  remote)))

          ;; Try to determine the upstream branch to track.
          (setq branch
                (or
                 ;; First try using explicitly configured branch in recipe.
                 (plist-get recalculated-recipe :branch)
                 ;; Then try getting remote's default branch.
                 (condition-case nil
                     (elpaca--remote-default-branch remote-str)
                   (error nil))))

          (when branch
            ;; Create a new local branch tracking the upstream.
            (elpaca--signal
             e (format "Creating %s branch tracking %s/%s" my/elpaca-branch-name remote-str branch))
            (elpaca-with-process-call
             ("git" "branch" my/elpaca-branch-name)
             (unless success
               (elpaca--signal
                e (format "Failed to create %s branch: %s" my/elpaca-branch-name stderr))))

            ;; Check out the branch.
            (elpaca-with-process-call
             ("git" "checkout" my/elpaca-branch-name)
             (unless success
               (elpaca--signal
                e (format "Failed to checkout %s branch: %s" my/elpaca-branch-name stderr))))

            ;; Set up tracking relationship.
            (elpaca-with-process-call
             ("git"
              "branch"
              "--set-upstream-to"
              (format "%s/%s" remote-str branch)
              my/elpaca-branch-name)
             (if success
                 (elpaca--signal
                  e (format "Successfully configured %s branch for updates" my/elpaca-branch-name))
               (elpaca--signal e (format "Failed to set upstream: %s" stderr)))))))))

  (elpaca--continue-build e))

;; Add the upstream configuration action to the main list of build steps, after the checkout-ref
;; step. These get executed in a number of contexts, for example when building or merging packages.
(let ((checkout-pos (cl-position 'elpaca--checkout-ref elpaca-build-steps)))
  (if checkout-pos
      (setq elpaca-build-steps
            (append
             (cl-subseq elpaca-build-steps 0 (1+ checkout-pos))
             (list 'my--elpaca-configure-upstream)
             (cl-subseq elpaca-build-steps (1+ checkout-pos))))
    (error "Could not find index to insert elpaca upstream configuration step")))

(defun my--elpaca-unpin (id)
  "Remove pinning keywords from package with ID.
Returns the modified elpaca struct."
  (when-let* ((e (elpaca-get id))
              (recipe (elpaca<-recipe e)))
    (dolist (keyword '(:pin :ref :tag))
      (when (plist-member recipe keyword)
        (setf (elpaca<-recipe e)
              (let ((copy (copy-tree recipe)))
                (cl-loop
                 for
                 (key val)
                 on
                 copy
                 by
                 #'cddr
                 unless
                 (memq key '(:pin :ref :tag))
                 collect
                 key
                 and
                 collect
                 val)))
        (elpaca--signal e (format "Removed %S from recipe" keyword) nil nil 1)))
    e))

;; Commands to remove pins from packages (i.e. the ref from the lockfile) to allow proper
;; fetching/merging/updating, which is normally impossible for locked packages.
;;
;; These are one-way commands, i.e. there's no way to re-pin a package once it's been unpinned
;; (except by restarting Emacs). In practice this doesn't really matter, as the pinning info is only
;; relevant when initially pulling the package.
(defun my/elpaca-unpin-all ()
  "Remove pins from all queued elpaca packages."
  (interactive)
  (cl-loop
   for
   (_ . e)
   in
   (elpaca--queued)
   unless
   (member (elpaca<-id e) elpaca-ignored-dependencies)
   do
   (my--elpaca-unpin (elpaca<-id e))))

(defun my/elpaca-unpin (id)
  "Remove pins from package with ID."
  (interactive (list (elpaca--read-queued "Unpin Package: ")))
  (my--elpaca-unpin id))

[petergaultney]

Can we do the opposite as things are currently designed? Manually write the lockfile when we feel like it, and have a separate, manual function to update all packages to match the lockfile? Using the variable seems a little inelegant somehow - it's declarative, but makes it so that you can no longer mark packages for upgrade at all.

I am probably missing an obvious elispism that would work to perform the "install from this lockfile" concept using the elpaca-lock-file variable.

[Martinsos]

Can we do the opposite as things are currently designed? Manually write the lockfile when we feel like it, and have a separate, manual function to update all packages to match the lockfile? Using the variable seems a little inelegant somehow - it's declarative, but makes it so that you can no longer mark packages for upgrade at all.

I am probably missing an obvious elispism that would work to perform the "install from this lockfile" concept using the elpaca-lock-file variable.

This is kind of also what I expectd -> I just realized now this is not the behaviour and so ended up on this issue. I believe my expectations are coming from the JS ecosystem, from npm -> there, package-lock.json is respected during installation of the packages only when that is explicitly demanded (via npm ci). This means that packages can be updated via the normal workflow, and package-lock.json is also automatically updated on any upgrade/install.

I am not sure what it he best experience for elpaca, for now probably the best course of action is just making it clear in the docs what is the current situation + how to go about it manually, and that is already great. As for some final DX -> npm style would probably make sense, also I probably wouldn't even mind the opposite, where I have to be explicit about not using the lock file, so I can pull in newer versions of packages.

Also, I understand that you implemented the current version of lock file support due to popular demand even though you had limited time to work on it, and I appreciate that a lot as it unlocks the whole thing and we can even have this discussion now :). The reason I wrote this comment here is merely as a feedback for the future, not an expectation or a demand.