copier-org
Overwrite does not apply
Describe the problem
When performing a copier update on a fresh repository, copier prompts to overwrite files, but doesn't actually overwrite them.
Template
https://github.com/andrew-glenn/example-copier-template
To Reproduce
❯ mkdir s32
❯ cd s32
❯ copier copy gh:andrew-glenn/example-copier-template $(pwd)
No git tags found in template; using HEAD as ref
Copying from template version 0.0.0.post1.dev0+9be7fdc
identical .
create example-file.txt
create LICENSE
create .copier-answers.yml
❯ date > example-file.txt
❯ git init . && git add -A . && git commit -m1
Initialized empty Git repository in /private/tmp/s32/.git/
[main (root-commit) 53f6c0b] 1
3 files changed, 206 insertions(+)
create mode 100644 .copier-answers.yml
create mode 100644 LICENSE
create mode 100644 example-file.txt
❯ copier update
No git tags found in template; using HEAD as ref
Updating to template version 0.0.0.post1.dev0+9be7fdc
No git tags found in template; using HEAD as ref
Copying from template version 0.0.0.post1.dev0+9be7fdc
identical .
conflict example-file.txt
Overwrite example-file.txt? [Y/n] Y
identical LICENSE
identical .copier-answers.yml
❯ cat example-file.txt
Fri Aug 12 14:45:44 CDT 2022
Expected behavior Copier overwrites the file it prompts me for.
Environment
- OS: OSX 12.5
- Copier version:
copier 6.1.0 - Python version:
Python 3.9.10 - Installation method: pypi
Additional context
I wrapped git to log the command it was performing. The following logs correspond to this code-block. I've truncated the path of the tempfiles for readability.
Code:
https://github.com/copier-org/copier/blob/141ddf1508eb89ae2380518edbfeae4edd0c14f5/copier/main.py#L697-L708
Logs:
git -C (...)/copier.main.update_diff.2od63kww init
git -C (...)/copier.main.update_diff.2od63kww add .
git -C (...)/copier.main.update_diff.2od63kww config user.name Copier
git -C (...)/copier.main.update_diff.2od63kww config user.email copier@copier
git -C (...)/copier.main.update_diff.2od63kww commit --allow-empty -am dumb commit 1
git -C (...)/copier.main.update_diff.2od63kww commit --allow-empty -am dumb commit 2
git -C (...)/copier.main.update_diff.2od63kww config --unset user.name
git -C (...)/copier.main.update_diff.2od63kww config --unset user.email
git -C (...)/copier.main.update_diff.2od63kww remote add real_dst file:///private/tmp/s32
git -C (...)/copier.main.update_diff.2od63kww fetch --depth=1 real_dst HEAD
git -C (...)/copier.main.update_diff.2od63kww diff-tree --unified=1 HEAD...FETCH_HEAD --inter-hunk-context=-1
Running the git diff at the end shows that HEAD is the upstream template, and FETCH_HEAD is the local project.
❯ git -C (...)/copier.main.update_diff.2od63kww diff-tree --unified=1 HEAD...FETCH_HEAD --inter-hunk-context=-1
diff --git a/example-file.txt b/example-file.txt
index 257cc56..59cd8b3 100644
--- a/example-file.txt
+++ b/example-file.txt
@@ -1 +1 @@
-foo
+Fri Aug 12 14:45:44 CDT 2022
In the case of an overwrite, I believe the order should be reversed, so that HEAD overwrites FETCH_HEAD, not maintaining the status-quo, and negating the affirmation to overwrite the file from upstream template.
Am I missing something?
Also - I'm more than happy to submit a PR if it's determined this is not the intended behavior. Just looking to brainstorm and verify I'm not missing anything here, as well as determine a path forward.
Hi! Thanks for the report and sorry for taking so long to answer. Following your steps I managed to reproduce the problem.
First of all, we have to keep in mind Copier is behaving "correctly". I mean that it is supposed to respect your changes in example-file.txt.
The smart update process is "smart" because it will only update something if it has changed in the template. Otherwise, Copier understands that you changed that file in your project because you had to, and live happy with that.
So, if your expectations are that Copier should put foo into example-file.txt, then please read carefully how the update process works. To achieve that, you should recopy the project instead of updating:
copier copy gh:andrew-glenn/example-copier-template .
However, there's a real bug: Copier shouldn't ask you to overwrite something that it's not going to overwrite.
The problem comes from a design dated before the smart update system was introduced in Copier 3.
This fix is to drop this "overwrite file?" form, as explained in https://github.com/copier-org/copier/issues/343#issuecomment-1163126904. Basically asking the user is useless in updates, because of 2 reasons:
- It's buggy (you saw this).
- Git is better deciding what changes to keep or discard than that form.
Also it's useless in copies because you're supposed to be copying something for the 1st time, so you should want everything overwritten.
So copier should overwrite always, period.
The tasks to do would be something around this:
- Remove the "overwrite" option, as it is unnecessary. Some code to remove would probably be: https://github.com/copier-org/copier/blob/fc6bb03a6db7e972742b812bb87e4330f58e3abc/copier/main.py#L145 https://github.com/copier-org/copier/blob/fc6bb03a6db7e972742b812bb87e4330f58e3abc/copier/main.py#L223-L254 https://github.com/copier-org/copier/blob/fc6bb03a6db7e972742b812bb87e4330f58e3abc/copier/cli.py#L163-L166
- Change lots of tests 😆
I'm more than happy to submit a PR
That'd be great! But let me advice you this is not easy task, as this feature is used very often. Do you still want to do it?
@yajo What's the status of this as I've the same problem for OCA repos?
For now, nothing changed apart from what I already explained in https://github.com/copier-org/copier/issues/741#issuecomment-1229394164.
The important part is that Copier is behaving correctly. It's just a UX problem, and the plan is to remove the overwrite question and just overwrite always. After all, updates are only allowed in git-tracked projects, and you can always review the diff afterwards with git, which is more comfortable and powerful than anything else I could possibly do.
As a workaround, you can pass --overwrite in CLI, to avoid the UX problem.