borg checkout includes ignored files when checking for uncommitted changes

[Thaodan]

When checking if a drone is modified ignored files are included as well.

Repositories containing byte compiled lisp or autoloads file are shown as modified even if they are not modified.

[tarsius]

When checking if a drone is modified

Which command/function are you talking about here? A Borg-specific function or some submodule related functionality provided by Magit?

[Thaodan]

I'm referring to borg.sh checkout command/function.

[tarsius]

So this:

checkout () {
...
        elif [ -n "$(git status --porcelain=v1 --ignored)" ]
        then
            echo "Skipping $path (uncommitted changes)"
...

Since I had to explicitly add the --ignored to make that happen, we can assume this is intentional. Unfortunately cd95c7cf55a9cab44a9cbd118045f2fec346dd17 was just a quick measure to prevent data from being eaten, and the message does not provide any justification for the use of --ignored.

It does make sense though. The goal is to prevent anything "the user has already done" from being discarded. The user may have run into issues and the attempts to address that may have resulted in the creation of ignored files, which we should avoid destroying. One may argue that this is very unlikely to happen, but I have never-the-less opted for the safe option here. Users can always explicitly (and forcefully, if needed) checkout a specific commit. Or they can git clean ... first.

[Thaodan]

Since I had to explicitly add the --ignored to make that happen, we can assume this is intentional. Unfortunately cd95c7cf55a9cab44a9cbd118045f2fec346dd17 was just a quick measure to prevent data from being eaten, and the message does not provide any justification for the use of --ignored.

Would this not imply that for ignored files there are no uncommitted changes? After all uncommitted changes affected files which are already tracked or staged.

It does make sense though. The goal is to prevent anything "the user has already done" from being discarded.

I get that point but that doesn't make much sense when it comes to files stored alongside the sources. Since we don't build drones out of tree the source will always be considered dirty when including ignored files.

[tarsius]

I'll have to experiment and it probably will take a while until I get to that.

[Thaodan]

No problem. Thanks for your work.

[higuoxing]

I'm facing a very similar issue while using borg. During compilation, byte codes are generated in submodules and the doc recommands ignoring these files in .config/git/ignore. I'm wondering if borg can provide some configuration variables to ignore dirty contents during assimilation?

During package compilation you may notice the submodules relating to
those packages become dirty due to the compilation outputs not being
ignored in those submodules.  For this reason it is useful to ignore
these outputs globally, for example in your ~~/.config/git/ignore~
file:

#+begin_src undefined
  *.elc
  *-autoloads.el
  dir
#+end_src

You may discover more things that you'll want to ignore this way as
you use ~borg~.

I did some attempts and here's a working demo.

---
 borg.el | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/borg.el b/borg.el
index 5046dd37a6c8..d75d9a719fda 100644
--- a/borg.el
+++ b/borg.el
@@ -1086,6 +1086,14 @@ non-nil, then try those files instead."
         (delete-file elc)))))

 ;;; Assimilation
+(defcustom borg-submodule-content-ignore-strategy
+  'none
+  "Specify the ignore strategy for submodules."
+  :type '(choice (const :tag "none" none)
+                 (const :tag "all" all)
+                 (const :tag "dirty" dirty)
+                 (const :tag "untracked" untracked))
+  :group 'some-group)

 (defun borg-assimilate (package url &optional partially)
   "Assimilate the package named PACKAGE from URL.
@@ -1106,6 +1114,8 @@ build and activate the drone."
     (borg--maybe-reuse-gitdir package)
     (borg--call-git package "submodule" "add" "--name" package url
                     (file-relative-name (borg-worktree package)))
+    (borg--call-git package "config" "--file" ".gitmodules"
+                    (format "submodule.%s.ignore" package) (symbol-name borg-submodule-content-ignore-strategy))
     (borg--sort-submodule-sections ".gitmodules")
     (borg--call-git package "add" ".gitmodules")
     (borg--maybe-absorb-gitdir package))
--
2.52.0

[Thaodan]

I think such an option would make sense if borg has an option to keep repositories pristine by building into a separate directory instead of the source tree.

But without such a thing no idea.