Template update process on Copier 9.3.0/9.3.1 is slow/indefinite on Windows compared with 9.2.0 for repos with lots of `.gitignore`s (I suspect a new `copytree` operation)

[blakeNaccarato]

Describe the problem

My template https://github.com/blakeNaccarato/copier-python is used for my project https://github.com/softboiler/boilercv, among others. Since Copier 9.3.0, I notice behavior where the cached repo contents in %TEMP% during e.g. copier update --vcs-ref=HEAD is 17GB+, and includes even .gitignored content such as .venv, and in this case, a .dvc/cache directory containing ~2000 files.

I think that this manifests as Copier trying to update from template and somehow starting to make an insanely large diff against the 17GB+ directory copied to %TEMP%. It causes a long-lasting hang during the template updating phase, which was at least fifteen minutes long before I cancelled the operation.

In Copier 9.2.0 and prior, I see the cached repo contents over in %TEMP% is only ~1GB, as expected, and doesn't venture into .gitignored territory. In comparing diffs across your tagged releases, I see the following section where a copytree operation seems to take place, only ignoring the .git folder. My suspicion is that this may be involved. See below for the permalink to the potentially offending code snippet.

By updating the template with Copier 9.2.0 instead of 9.3.0 or 9.3.1, I get the relatively fast template update operation that I expect.

https://github.com/copier-org/copier/blob/26c811f2f3152549c9f5d908cba79f142c0eccd6/copier/main.py#L936-L948

Template

https://github.com/blakeNaccarato/copier-python

To Reproduce

1. Potentially, be on Windows, but this may be OS agnostic
2. Have a repository with lots and lots of .gitignored files (perhaps "lots" by number of files, or by size on disk, or both)
3. Run copier update on this repository, with Copier 9.3.0 or 9.3.1
4. Wait forever, copier update will not finish

Logs

No response

Expected behavior

Template updating should behave as in Copier 9.2.0, with no major slowdown and no massive copytree operation sending ~17GB to %TEMP%.

Screenshots/screencasts/logs

No response

Operating system

Windows

Operating system distribution and version

Windows 11

Copier version

9.3.0

Python version

CPython 3.11

Installation method

pipx+pypi

Additional context

No response

[sisp]

Thanks for reporting this, @blakeNaccarato! It's a performance regression, you already traced it back correctly. I'll look into it asap. Sorry for the inconvenience.

[pawamoy]

Now that you mention it, I noticed this slowness too on Linux since 9.3.0.

[blakeNaccarato]

No worries @sisp, glad I could point in the right direction! Regarding inconvenience, I think Copier must have saved me hundreds of hours of inconvenience over the past couple years, so I'm not too worried about a little inconvenience caused. Thanks!