dvc
dvc copied to clipboard
iterative
[Windows] Network drive - Unexpected error - path is on mount ...
Hi,
I work on Windows in corporate environment on network drive, so my working directory in explorer looks like:
Home (\aaa.bbb.ccc.net\UserData\User1) (P:)
In conda prompt it look like ordinary P: folder. I create folder like test_DVC. In this folder: git init, dvc init works fine, but when I do dvc add somefile.txt, I get:
ERROR: Unexpected error - path is on mount '\\\\aaa.bbb.ccc.net\\UserData, start on '\\\\AD3AB.AAA.BBB.CCC.NET\\UserData'
I've tried also:
dvc add P:\test_DVC\somefile.txt, I get:
ERROR: Output(s) outside of DVC project: P:\test_DVC\somefile.txt
I've also tried without git, by dvc init --no-scm but result is the same
DVC version: 2.11.0 Is it expected and I miss something obvious, or bug?
Can you please run dvc doctor and post the output, as well as re-trying the add command with verbose output (dvc add -v) and including the full error log.
This is likely the same issue as https://github.com/iterative/dvc/issues/7909 (the URL in UNC paths should also be case-insensitive)
@pmrowla I think it is not only about case-sensitivity, there is an additional element in path: '\\aaa.bbb.ccc.net\UserData, start on '\\AD3AB.AAA.BBB.CCC.NET\UserData'
Traceback of dvc add file.csv (with some obfuscation):
2022-06-23 09:14:43,182 ERROR: unexpected error - path is on mount '\\\\[aaa.bbb.net](http://aaa.bbb.net/)\\UserData', start on mount '\\\\NLDN…[PAU.AAA.BBB.NET](http://pau.aaa.bbb.net/)\\UserData'
------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\user..\AppData\Roaming\Python\Python38\site-packages\dvc\cli\__init__.py", line 185, in main
ret = cmd.do_run()
File "C:\Users\ user..\AppData\Roaming\Python\Python38\site-packages\dvc\cli\command.py", line 22, in do_run
return self.run()
File "C:\Users\ user..\AppData\Roaming\Python\Python38\site-packages\dvc\commands\add.py", line 22, in run
self.repo.add(
File "C:\Users\ user..\AppData\Roaming\Python\Python38\site-packages\dvc\utils\collections.py", line 152, in inner
result = func(*ba.args, **ba.kwargs)
File "C:\Users\glowacm\AppData\Roaming\Python\Python38\site-packages\dvc\repo\__init__.py", line 49, in wrapper
return f(repo, *args, **kwargs)
File "C:\Users\ user..\AppData\Roaming\Python\Python38\site-packages\dvc\repo\scm_context.py", line 152, in run
return method(repo, *args, **kw)
File "C:\Users\glowacm\AppData\Roaming\Python\Python38\site-packages\dvc\repo\add.py", line 176, in add
new_index = repo.index.update(stages)
File "C:\Users\ user..\AppData\Roaming\Python\Python38\site-packages\dvc\repo\index.py", line 240, in update
new_stages = set(stages)
File "C:\Users\glowacm\AppData\Roaming\Python\Python38\site-packages\dvc\stage\__init__.py", line 210, in __hash__
return hash(self.path_in_repo)
File "C:\ProgramData\Miniconda3\envs\py38NLG\lib\site-packages\funcy\objects.py", line 28, in __get__
res = instance.__dict__[self.fget.__name__] = self.fget(instance)
File "C:\Users\glowacm\AppData\Roaming\Python\Python38\site-packages\dvc\stage\__init__.py", line 221, in path_in_repo
return relpath(self.path, self.repo.root_dir)
File "C:\Users\ user..\AppData\Roaming\Python\Python38\site-packages\dvc\utils\__init__.py", line 255, in relpath
return os.path.relpath(path, start)
File "C:\ProgramData\Miniconda3\envs\py38NLG\lib\ntpath.py", line 703, in relpath
raise ValueError("path is on mount %r, start on mount %r" % (
ValueError: path is on mount '\\\\[aaa.bbb.net](http://aaa.bbb.net/)\\UserData', start on mount '\\\\NLDN…[PAU.AAA.BBB.NET](http://pau.aaa.bbb.net/)\\UserData'
------------------------------------------------------------
2022-06-23 09:14:44,046 DEBUG: Version info for developers:
DVC version: 2.11.0 (pip)
---------------------------------
Platform: Python 3.8.12 on Windows-10-10.0.19042-SP0
Supports:
webhdfs (fsspec = 2021.11.0),
http (aiohttp = 3.8.1, aiohttp-retry = 2.4.8),
https (aiohttp = 3.8.1, aiohttp-retry = 2.4.8)
Cache types: https://error.dvc.org/no-dvc-cache
Caches: local
Remotes: None
Workspace directory: NTFS on P:\
Repo: dvc (no_scm)
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2022-06-23 09:14:44,054 DEBUG: Analytics is enabled.
2022-06-23 09:14:44,303 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', 'C:\\Users\\user…\\AppData\\Local\\Temp\\2\\tmp2ob7xl45']'
2022-06-23 09:14:44,321 DEBUG: Spawned '['daemon', '-q', 'analytics', 'C:\\Users\\user…\\AppData\\Local\\Temp\\2\\tmp2ob7xl45']'
@mglowacki100 are you able to try changing your mapped network drive so that you mount \\AD3AB.AAA.BBB.CCC.NET\... (instead of the shortened \\AAA.BBB.CCC.net\...)? It looks like your network/system may configured so that the shorter hostname is always resolved into the full AD3... hostname in UNC paths, which causes issues when DVC tries to compare the two
@pmrowla Thanks. I've created another drive with full path and it seems to work fine there :) I've heard that this prefix "AD3AB" changes from time to time, would be enough to update mapping or "dvc repo" would be corrupted in some way?
As long as the mapping is up to date whenever you use dvc add it should be ok. The DVC repo itself does not store anything regarding absolute paths, the problem is just during a verification step during dvc add.
We can keep this issue open and try to investigate whether or not it's possible to work around the automatic host resolution, but it likely won't be prioritized right away since it's a windows and system-configuration specific problem.
Thanks for help! Maybe enough is to add this fix in help/documentation.
Hia - I've run into this same error, also on Windows:
Z:\path\to\project> dvc repro -v
2022-08-23 17:28:36,413 DEBUG: Lockfile for 'dvc.yaml' not found
2022-08-23 17:28:36,427 ERROR: unexpected error - path is on mount '\\\\<network-drive>\\<folder>', start on mount 'Z:'
------------------------------------------------------------
Traceback (most recent call last):
File "dvc\cli\__init__.py", line 185, in main
File "dvc\cli\command.py", line 22, in do_run
File "dvc\commands\repro.py", line 13, in run
File "dvc\repo\__init__.py", line 48, in wrapper
File "dvc\repo\scm_context.py", line 156, in run
File "dvc\repo\reproduce.py", line 145, in reproduce
File "funcy\objects.py", line 28, in __get__
File "dvc\repo\index.py", line 162, in graph
File "funcy\objects.py", line 28, in __get__
File "dvc\repo\index.py", line 74, in stages
File "dvc\repo\stage.py", line 516, in collect_repo
File "dvc\repo\stage.py", line 499, in _collect_repo
File "dvc\repo\stage.py", line 319, in load_file
File "dvc\repo\stage.py", line 299, in load_all
File "dvc\repo\stage.py", line 299, in <listcomp>
File "_collections_abc.py", line 720, in __iter__
File "dvc\stage\loader.py", line 152, in __iter__
File "funcy\objects.py", line 28, in __get__
File "dvc\stage\loader.py", line 39, in resolver
File "dvc\parsing\__init__.py", line 140, in __init__
File "dvc_objects\fs\path.py", line 123, in relpath
File "ntpath.py", line 703, in relpath
ValueError: path is on mount '\\\\<network-drive>\\<folder>', start on mount 'Z:'
------------------------------------------------------------
2022-08-23 17:28:39,519 DEBUG: Version info for developers:
DVC version: 2.18.0 (exe)
---------------------------------
Platform: Python 3.8.10 on Windows-10-10.0.17763-SP0
Supports:
azure (adlfs = 2022.7.0, knack = 0.9.0, azure-identity = 1.10.0),
gdrive (pydrive2 = 1.14.0),
gs (gcsfs = 2022.7.1),
hdfs (fsspec = 2022.7.1, pyarrow = 9.0.0),
webhdfs (fsspec = 2022.7.1),
http (aiohttp = 3.8.1, aiohttp-retry = 2.8.3),
https (aiohttp = 3.8.1, aiohttp-retry = 2.8.3),
s3 (s3fs = 2022.7.1, boto3 = 1.21.21),
ssh (sshfs = 2022.6.0),
oss (ossfs = 2021.8.0),
webdav (webdav4 = 0.9.7),
webdavs (webdav4 = 0.9.7)
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: None
Workspace directory: NTFS on Z:\
Repo: dvc, git
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2022-08-23 17:28:39,670 DEBUG: Analytics is enabled.
2022-08-23 17:28:39,675 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', 'C:\\Users\\<user>\\AppData\\Local\\Temp\\7\\tmpaqo4lrof']'
2022-08-23 17:28:39,685 DEBUG: Spawned '['daemon', '-q', 'analytics', 'C:\\Users\\<user>\\AppData\\Local\\Temp\\7\\tmpaqo4lrof']'
@mglowacki100 - could you please elaborate how you resolved this issue? I note there are slight differences in the format of our UNC addresses, but I haven't been able to resolve this by re-mapping the network drive.
@a-orn It was some time ago , but IIRC:
- You need to add drive with full path, beware that "hidden parts" like to change from time to time by sys admins and this fix stop to work, (update of full drive path is required). Let's call this new drive N:
- In command line you need to change drive without hidden parts to N
- next create test dir: mkdir dvc_test, cd dvc_test, git init
- now dvc init probably will fail (dvc is not recognized as internal or external command..). You need to locate dvc.exe, if you installed it with conda it should be in conda folder ....\envs\your_env\Scripts\dvc.exe, so you can run this as absolute path C:...\dvc.exe init or make a shortcut
- now dvc add sample_file.txt should work (shortcut version)
If you've this basic test configuration up and running, then check your specific case. Hope this helps.