dvc
dvc copied to clipboard
dvc.exceptions.CyclicGraphError: Pipeline has a cycle involving: load_extract_save.
stages:
load_extract_save:
cmd: python src/stage_01_load_extract_save.py --config=config/config.yaml
deps:
- config/config.yaml
- src/utils/all_utils.py
- src/stage_01_load_extract_save.py
- artifacts/data
outs:
- artifacts/data
- artifacts/clean_data/X.npy
- artifacts/clean_data/Y.npy
train_test_split_save:
cmd: python src/stage_02_train_test_split_save.py --config=config/config.yaml --params=params.yaml
deps:
- artifacts/clean_data/X.npy
- artifacts/clean_data/Y.npy
- src/utils/all_utils.py
- params.yaml
- config/config.yaml
- src/stage_02_train_test_split_save.py
outs:
- artifacts/train_data/X_train.npy
- artifacts/train_data/Y_train.npy
- artifacts/test_data/X_test.npy
- artifacts/test_data/Y_test.npy
train_model:
cmd: python src/stage_03_train.py --config=config/config.yaml --params=params.yaml
deps:
- artifacts/train_data/X_train.npy
- artifacts/train_data/Y_train.npy
- artifacts/test_data/X_test.npy
- artifacts/test_data/Y_test.npy
- src/stage_03_train.py
- src/utils/all_utils.py
- config/config.yaml
- params.yaml
outs:
- artifacts/checkpoints
- artifacts/model
metrics:
cmd: python src/stage_04_metrics.py --config=config/config.yaml
deps:
- src/stage_04_metrics.py
- config/config.yaml
- src/utils/all_utils.py
- artifacts/test_data/X_test.npy
- artifacts/test_data/Y_test.npy
- artifacts/checkpoints
- artifacts/model
outs:
- confusion_matrix.png
This is my dvc.yaml.
I have created github workflows, but the run fails and is giving me this error - 2022-06-17 21:34:59,843 ERROR: Pipeline has a cycle involving: load_extract_save.
I have tried down grading the dvc module but nothing works.
You have artifacts/data listed as both a dependency and an output in that stage, which is not supported in DVC (due to being a cyclic/circular dependency)
Thanks a lot, it worked😁
Thanks for clarifying @pmrowla!
It might be worth putting this clarification into the error message itself e.g.
ERROR: same item(s) defined as both a dependency and an output in stage: <stage name>
Reopening as a Good First Issue for the UI enhancement.
Would need to update
https://github.com/iterative/dvc/blob/f7fbac1eed12c8e5e7b6657fd741d4f426a6fe26/dvc/exceptions.py#L132-L138
Used in https://github.com/iterative/dvc/blob/main/dvc/repo/graph.py#L27