vscode-dvc icon indicating copy to clipboard operation
vscode-dvc copied to clipboard
verified publisher icon iterative

DVC tracked file rename fails when file is not pulled yet

Open shcheklein opened this issue 4 years ago • 3 comments

After trying to rename the images in the get-started-experiment:

Screen Shot 2021-09-13 at 10 41 58 PM

File and directory are completely missing in the DVC tracked view.

Logs ``` ERROR: failed to get diff - Unable to acquire lock. Most likely another DVC process is running or was terminated abruptly. Check the page for other possible reasons and to learn how to resolve this. [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:30:10.240Z, pid: 8062] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc status --show-json - COMPLETED (7189ms) [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:30:34.872Z, pid: 8071] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . --dvc-only -R --show-json - COMPLETED (31813ms) [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:34:09.044Z, pid: 9486] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . data --dvc-only --show-json - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:34:09.572Z, pid: 9486] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . data --dvc-only --show-json - COMPLETED (530ms) [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:35:26.889Z, pid: 10054] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc pull data/images.tar.gz - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:35:29.514Z, pid: 10054] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc pull data/images.tar.gz - FAILED with code 255 (2629ms) ERROR: unexpected error - [Errno 22] The provided token is malformed or otherwise invalid.: An error occurred (InvalidToken) when calling the ListObjectsV2 operation: The provided token is malformed or otherwise invalid. [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:39:06.877Z, pid: 11773] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc diff --show-json - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:39:06.879Z, pid: 11774] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc status --show-json - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:39:06.887Z, pid: 11783] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . --dvc-only -R --show-json - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:39:09.817Z, pid: 11773] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc diff --show-json - FAILED with code 1 (2944ms) ERROR: failed to get diff - Unable to acquire lock. Most likely another DVC process is running or was terminated abruptly. Check the page for other possible reasons and to learn how to resolve this. [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:39:14.659Z, pid: 11774] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc status --show-json - COMPLETED (7782ms) [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:40:05.360Z, pid: 11783] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . --dvc-only -R --show-json - COMPLETED (58476ms) [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:40:18.505Z, pid: 12299] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc move data/images.tar.gz data/images1.tar.gz - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:40:19.518Z, pid: 12326] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . data --dvc-only --show-json - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:40:20.648Z, pid: 12326] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . data --dvc-only --show-json - FAILED with code 1 (1132ms) ERROR: failed to list '.' - The path 'data' does not exist in the target repository '/Users/ivan/Projects/get-started-experiments' as an DVC output. [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:40:20.681Z, pid: 12331] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . --dvc-only --show-json - INITIALIZED [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:40:21.427Z, pid: 12299] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc move data/images.tar.gz data/images1.tar.gz - FAILED with code 255 (2925ms) ERROR: unexpected error - [Errno 2] No such file or directory: '/Users/ivan/Projects/get-started-experiments/data/images.tar.gz' [version: 0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0, 2021-09-14T05:40:21.938Z, pid: 12331] > /Users/ivan/Projects/get-started-experiments/.env/bin/python -m dvc list . --dvc-only --show-json - COMPLETED (1260ms) ```
Environment

0.1.0-e1bf948c5cc02edf6260c675401d500502abe9b0

Version: 1.60.0
Commit: e7d7e9a9348e6a8cc8c03f877d39cb72e5dfb1ff
Date: 2021-09-01T10:54:53.442Z
Electron: 13.1.8
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Darwin x64 19.6.0

DVC version: 2.7.3+788b4d 
---------------------------------
Platform: Python 3.8.9 on macOS-10.15.6-x86_64-i386-64bit
Supports:
        azure (adlfs = 2021.9.1, knack = 0.8.2, azure-identity = 1.6.0),
        gdrive (pydrive2 = 1.9.3),
        gs (gcsfs = 2021.8.1),
        hdfs (pyarrow = 4.0.1),
        webhdfs (hdfs = 2.6.0),
        http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
        https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
        s3 (s3fs = 2021.8.1, boto3 = 1.17.106),
        ssh (sshfs = 2021.8.1),
        oss (ossfs = 2021.8.0),
        webdav (webdav4 = 0.9.1),
        webdavs (webdav4 = 0.9.1)
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk1s1
Caches: local
Remotes: s3
Workspace directory: apfs on /dev/disk1s1
Repo: dvc, git

shcheklein avatar Sep 14 '21 05:09 shcheklein

Overall, rename command is very complicated in DVC and move doesn't implement that semantics properly. It's another item that we'll need to discuss and prioritize on the DVC end (cc @efiop @dberenbaum ).

We'll need to being able to rename model files (outputs of the pipelines). Or don't allow that action in the first place as a simple workaround.

shcheklein avatar Sep 14 '21 05:09 shcheklein

Relates to https://github.com/iterative/vscode-dvc/issues/569

mattseddon avatar Sep 14 '21 09:09 mattseddon

First workaround to try: disable this in situations that DVC can't handle (e.g. when files are not physically present yet).

shcheklein avatar Aug 23 '22 23:08 shcheklein