dvc
dvc copied to clipboard
dvc pull with multiple .dvc files into single output folder
Hello. I have some folder that is already on DVC. It is stored locally like that:
data/release_123/folder_1.dvc
folder_1 has following structure:
data/release_123/folder_1/subfolder_1 data/release_123/folder_1/subfolder_2
So when I run dvc pull data/release_123/folder_1.dvc
a folder appears locally: data/release_123/folder_1.
Now I want to add some new folders to folder_1, but also I want to have two separate dvc files for that. For example, lets name it data/release_123/folder_1_new_subfolders.dvc.
I want it work like that:
- When I run
dvc pull data/release_123/folder_1.dvc
, old folder structure appears (like the one above). - When I run
dvc pull data/release_123/folder_1_new_subfolders.dvc
, newly added folders should be pulled into folder_1.
Final structure that I expect to appear after running 1 and 2 (actually not depending on order):
data/release_123/folder_1/subfolder_1 data/release_123/folder_1/subfolder_2 data/release_123/folder_1/subfolder_3 data/release_123/folder_1/subfolder_4
How to do that? I've tried the following:
- Put new subfolders to other folder:
data/release_123/folder_2/subfolder_3 data/release_123/folder_2/subfolder_4
-
dvc add data/release_123/folder_2
-
dvc push
- Manually renaming "path" field in data/release_123/folder_2.dvc from folder_2 to folder_1.
But in this case, I always get this structure:
data/release_123/folder_1/subfolder_3 data/release_123/folder_1/subfolder_4
when running any of dvc pull data/release_123/folder_1.dvc
, dvc pull data/release_123/folder_2.dvc
in any order.