Directories with `remove_` attribute are always somehow being changed since chezmoi last wrote it

[jakan0]

Describe the bug

I'm not sure if this is just a misunderstanding on my part on how the remove_ attribute is supposed to work or if it's a bug. What I'm trying to do is to not create empty directory paths when a template file in the path has no output.

Initially everything seems to be working fine, but when apply command is run for the second time, chezmoi complains that path has changed since chezmoi last wrote it. To me it seems like the internal database is not updated after empty directories with remove_ attribute are removed.

To reproduce

Create the following repository structure:

.
├── .chezmoi.toml.tmpl
└── remove_dot_foobar
    └── dot_nobackup.tmpl

Add the following content to the .chezmoi.toml.tmpl file.

[data]
nobackup = false

Add the following content to the dot_nobackup.tmpl file.

{{ if .nobackup -}}
# nobackup
{{ end -}}

Expected behavior

Based on the documentation I expected to see no empty directories being created, which kind of are not being created, and then the subsequent apply command to not complain about the path being changed.

Output of command with the --verbose flag

+ chezmoi init

+ chezmoi status
 A .foobar

+ chezmoi --verbose apply
diff --git a/.foobar b/.foobar
new file mode 40755
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
--- /dev/null
+++ b/.foobar
diff --git a/.foobar b/.foobar
deleted file mode 40755
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000
--- a/.foobar
+++ /dev/null

+ chezmoi status
DA .foobar

+ chezmoi --verbose apply
.foobar has changed since chezmoi last wrote it [overwrite,all-overwrite,skip,quit]? o
diff --git a/.foobar b/.foobar
new file mode 40755
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
--- /dev/null
+++ b/.foobar
diff --git a/.foobar b/.foobar
deleted file mode 40755
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000
--- a/.foobar
+++ /dev/null

+ chezmoi status
DA .foobar

Output of chezmoi doctor

+ chezmoi doctor
RESULT    CHECK                MESSAGE
ok        version              v2.17.1, commit 565cbbe117746aa6bfec5f2cee20ae4cbbb5e645, built at 2022-05-30T10:24:34Z, built by goreleaser
ok        latest-version       v2.17.1
ok        os-arch              linux/amd64 (Alpine Linux)
ok        uname                Linux db44928d96da 5.17.13-200.fc35.x86_64 #1 SMP PREEMPT Mon Jun 6 14:38:57 UTC 2022 x86_64 Linux
ok        go-version           go1.18.2 (gc)
ok        executable           ~/bin/chezmoi
ok        upgrade-method       replace-executable
ok        config-file          ~/.config/chezmoi/chezmoi.toml
ok        source-dir           ~/.local/share/chezmoi is a directory
ok        suspicious-entries   no suspicious entries
ok        working-tree         ~/.local/share/chezmoi is a directory
ok        dest-dir             ~ is a directory
ok        shell-command        found /bin/ash
ok        shell-args           /bin/ash
ok        cd-command           found /bin/ash
ok        cd-args              /bin/ash
ok        edit-command         found /usr/bin/vi
ok        edit-args            /usr/bin/vi
info      diff-command         not set
ok        umask                022
ok        git-command          found /usr/bin/git, version 2.36.1
warning   merge-command        vimdiff not found in $PATH
info      age-command          age not found in $PATH
info      gpg-command          gpg not found in $PATH
info      pinentry-command     not set
info      1password-command    op not found in $PATH
info      bitwarden-command    bw not found in $PATH
info      gopass-command       gopass not found in $PATH
info      keepassxc-command    keepassxc-cli not found in $PATH
info      keeper-command       keeper not found in $PATH
info      keepassxc-db         not set
info      lastpass-command     lpass not found in $PATH
info      pass-command         pass not found in $PATH
info      vault-command        vault not found in $PATH
info      secret-command       not set

Additional context

The output shown above is from a simplified example running in a Podman container for demonstration purposes but the same thing happens when executing this on the host. The host machine's output of chezmoi doctor is shown below.

+ chezmoi doctor
RESULT    CHECK                MESSAGE
ok        version              v2.15.4, commit bec3d0a03a0ccdcd2f4be319568d764eff0c2777, built at 2022-05-09T22:42:35Z, built by goreleaser
warning   latest-version       v2.17.1
ok        os-arch              linux/amd64 (Fedora Linux 35 (Workstation Edition))
ok        uname                Linux fedora 5.17.13-200.fc35.x86_64 #1 SMP PREEMPT Mon Jun 6 14:38:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
ok        go-version           go1.18.1 (gc)
ok        executable           ~/.local/bin/chezmoi
ok        upgrade-method       replace-executable
warning   config-file          ~/.config/chezmoi/chezmoi.toml and ~/.config/chezmoi/chezmoi.yaml: multiple config files
ok        source-dir           ~/.local/share/chezmoi is a directory
warning   suspicious-entries   ~/.local/share/chezmoi/tmp/private/chezmoi.toml.tmpl
ok        working-tree         ~/.local/share/chezmoi is a directory
ok        dest-dir             ~ is a directory
ok        shell-command        found /bin/sh
ok        shell-args           /bin/sh
ok        cd-command           found /bin/sh
ok        cd-args              /bin/sh
ok        edit-command         found /usr/bin/nano
ok        edit-args            /usr/bin/nano
info      diff-command         not set
ok        umask                022
ok        git-command          found /usr/bin/git, version 2.35.3
warning   merge-command        vimdiff not found in $PATH
info      age-command          age not found in $PATH
ok        gpg-command          found /usr/bin/gpg, version 2.3.4
info      pinentry-command     not set
info      1password-command    op not found in $PATH
ok        bitwarden-command    found ~/.local/bin/bw, version 1.9.1
info      gopass-command       gopass not found in $PATH
ok        keepassxc-command    found /usr/bin/keepassxc-cli, version 2.7.1
info      keepassxc-db         not set
info      lastpass-command     lpass not found in $PATH
info      pass-command         pass not found in $PATH
info      vault-command        vault not found in $PATH
info      secret-command       not set

[twpayne]

Thank you very much for reporting this with clear steps on how to reproduce the problem - much appreciated!

This is definitely a bug in chezmoi. I've added a test case to reproduce the problem in #2133.

[twpayne]

OK, so on further investigation this has revealed a rather subtle flaw in chezmoi that might take a while to fix. Roughly what seems to be happening is:

• For usability and performance, chezmoi only evaluates target states of files and directories when they are needed. The usability aspect means that the password manager/encryption is only invoked if it is needed to complete the command, so you don't get prompted to enter passwords/keys for operations that do not need them. The performance aspect is that chezmoi avoids computation for operations whose results will not be used.
• The special combination here is a remove_ directory containing a file that may or may not exist (the files contents are a template that might be empty).
• chezmoi's support for remove_ directories is currently a horrible hack. Specifically, chezmoi adds them to a list when it encounters them and then runs rmdir on each remove_ directory as a post-apply phase. This removes the directory if it contains no entries, or keeps if it contains at least one entry (whether or not that entry is managed by chezmoi).
• remove_ directories should be removed if they contain no entries. However, whether the contained file should exist or not can only be determined by evaluating all entries of that directory, which chezmoi does not do correctly.
• The .foobar has changed since chezmoi last wrote it error comes from chezmoi confusing itself because chezmoi records that it last wrote a directory, chezmoi doesn't record that chezmoi later removed that directory with rmdir, and so chezmoi thinks that someone else removed (changed) the directory.

Fixing this will require some effort. The short term fix is to remove the remove_ attribute from the .foobar directory (chezmoi chattr noremove ~/.foobar) and add a run_after_ script that removes the directory if it is empty (echo "#\!/bin/sh\nrmdir $HOME/.foobar" > $(chezmoi source-path)/run_after_remove-foobar-if-empty.sh).

Thanks again for your keen insight and reporting the issue - much appricated!

[joelanford]

I'm also running into this problem. The run_after workaround is only partially working for me though (perhaps I have a slightly different scenario?).

If I repeatedly run chezmoi apply on a machine where the directories are empty, they are removed as expected, but the next run of chezmoi apply complains, saying <removeDir> has changed since chezmoi last wrote it. I think this is because chezmoi has a recording of this directory existing in its state, but then the run_after script removes the directory, causing chezmoi to rightfully complain that things are out of sync.

I'm curious if part of the solution is to only record the <removeDir> in the entryState if it contains at least one file or directory that is present and managed by chezmoi?

[jakan0]

Thanks @twpayne for investigating this so quickly. I understand that the fix might not be an easy one. I'm currently refactoring my dotfiles repository to take advantage of the new features of chezmoi, so I think I'm just going to leave the remove_ attributes and just deal with the errors one by one when I apply the changes, because I don't want to have for example empty VS Code directories on headless servers.

[twpayne]

I don't want to have for example empty VS Code directories on headless servers.

For this you can use .chezmoiignore to not create the VS Code directory at all on headless servers.

[jakan0]

Yeah sorry, I know about .chezmoiignore, and I've been using it to manage for example my personal and work dotfiles in the same repo. VS Code was actually a really bad example because that won't end up on a headless server in the first place. A better example would be .bashrc and other Bash related files. Because those are created by default almost always, and I use Zsh as a shell whenever I can, I would like to get rid of Bash config files.

I also tried using a combination of .chezmoiignore and .chezmoiremove to get rid of those unwanted config files, but either I get e.g. chezmoi: .bashrc: inconsistent state (.chezmoiremove, private_dot_bashrc.tmpl) error even when the template outputs an empty file, or the unwanted files are completely ignored.