dvc icon indicating copy to clipboard operation
dvc copied to clipboard

dvc move: puts 3 lines for each moved file in destination folder/.gitignore on Windows Anaconda Prompt

Open printomi opened this issue 3 years ago • 3 comments

Bug Report

move: puts 3 lines for each moved file in destination folder/.gitignore on Windows Anaconda Prompt

Description

dvc move triples the lines for each moved file in the destination .gitignore on Windows, in Anaconda Prompt.

Reproduce

  1. Place a data file to data/data.xml (from the get started guide)
  2. Track it with DVC.
  3. Create another directory.
  4. Move data file to the new directory.
(dvc) D:\dvc\dvc_example\data>dir
 Volume in drive D has no label.
 Volume Serial Number is F48F-2DD0

 Directory of D:\dvc\dvc_example\data

2022. 06. 13.  12:16    <DIR>          .
2022. 06. 13.  12:16    <DIR>          ..
2022. 06. 13.  11:12                44 .gitignore
2022. 06. 02.  13:24        14 445 097 data.xml
2022. 06. 02.  15:12                84 data.xml.dvc
2022. 06. 02.  13:48    <DIR>          features
2022. 06. 13.  11:12             4 611 iris.csv
2022. 06. 13.  12:16                80 iris.csv.dvc
2022. 06. 02.  13:48    <DIR>          prepared
               5 File(s)     14 449 916 bytes
               4 Dir(s)  997 414 789 120 bytes free

(dvc) D:\dvc\dvc_example\data>type .gitignore
/data.xml
/prepared
/features
/iris.csv

(dvc) D:\dvc\dvc_example\data>mkdir data2

(dvc) D:\dvc\dvc_example\data>dvc move --verbose data.xml data2\data.xml
2022-06-16 15:00:02,082 DEBUG: Adding 'data2\data.xml' to 'data2\.gitignore'.
2022-06-16 15:00:02,154 DEBUG: {}
2022-06-16 15:00:02,157 DEBUG: Output 'data2\data.xml' didn't change. Skipping saving.
2022-06-16 15:00:02,164 DEBUG: Preparing to transfer data from '..\.dvc\cache' to '..\.dvc\cache'
2022-06-16 15:00:02,172 DEBUG: Cache type 'reflink' is not supported: reflink is not supported
2022-06-16 15:00:02,181 DEBUG: Created 'copy': ..\.dvc\cache\.cache_type_test_file -> data2\.chi7BP2dhKriARZayBXb33
2022-06-16 15:00:02,184 DEBUG: Removing 'data2\.chi7BP2dhKriARZayBXb33'
2022-06-16 15:00:02,187 DEBUG: Removing '..\.dvc\cache\.cache_type_test_file'
2022-06-16 15:00:02,191 DEBUG: Skipping copying for 'data2\data.xml', since it is not a symlink or a hardlink.
2022-06-16 15:00:02,193 DEBUG: state save (562949954085724, 1654169070734948352, 14445097) 22a1a2931c8370d3aeedd7183606fd7f
2022-06-16 15:00:02,358 DEBUG: state save (562949954085724, 1654169070734948352, 14445097) 22a1a2931c8370d3aeedd7183606fd7f
2022-06-16 15:00:02,366 DEBUG: Adding 'data2\data.xml' to 'data2\.gitignore'.
2022-06-16 15:00:02,383 DEBUG: Adding 'data2\data.xml' to 'data2\.gitignore'.
2022-06-16 15:00:02,393 DEBUG: {'data2\\data.xml': 'modified'}
2022-06-16 15:00:02,399 DEBUG: Computed stage: 'data2\data.xml.dvc' md5: 'None'
2022-06-16 15:00:02,403 DEBUG: Saving information to 'data2\data.xml.dvc'.

To track the changes with git, run:

        git add 'data2\.gitignore' .gitignore 'data2\data.xml.dvc'
2022-06-16 15:00:02,540 DEBUG: Analytics is enabled.
2022-06-16 15:00:02,757 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', 'C:\\Users\\FENYVE~1.FT\\AppData\\Local\\Temp\\tmp9o62w5aq']'
2022-06-16 15:00:02,789 DEBUG: Spawned '['daemon', '-q', 'analytics', 'C:\\Users\\FENYVE~1.FT\\AppData\\Local\\Temp\\tmp9o62w5aq']'

(dvc) D:\dvc\dvc_example\data>type data2\.gitignore
/data.xml
/data.xml
/data.xml

(dvc) D:\dvc\dvc_example\data>

Expected

I expected that the .gitignore file in the destination folder would contain the line \data.xml only once.

Environment information

Output of dvc doctor:

(dvc) D:\dvc\dvc_example\data>dvc doctor
DVC version: 2.8.1 (conda)
---------------------------------
Platform: Python 3.7.13 on Windows-10-10.0.19041-SP0
Supports:
        webhdfs (fsspec = 2021.10.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.4.5),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.4.5)
Cache types: hardlink
Cache directory: NTFS on D:\
Caches: local
Remotes: local
Workspace directory: NTFS on D:\
Repo: dvc, git

Additional Information (if any):

I use Conda installation, but I have just installed the standalone distribution to try the VS Code extension, because VisualStudio Code didn't see the Conda installation. The standalone installer told me to re-login to apply a new privilege, but I have not logged in since then.

printomi avatar Jun 16 '22 13:06 printomi

@skshetry Maybe you have any idea why this is happening?

dberenbaum avatar Jun 29 '22 17:06 dberenbaum

This sounds very similar to other dulwich gitignore issues (i.e. #7815)

daavoo avatar Jun 30 '22 08:06 daavoo

This sounds very similar to other dulwich gitignore issues (i.e. #7815)

Yeah, but I think this one is a Windows special one. The problem of #7815 is compare of data/ and data, Here I guess the problem comes from \data.xml and /data.xml or maybe an invisible ending character.

karajan1001 avatar Jul 05 '22 06:07 karajan1001