Error when pipeline.yaml file has indentation issue
Pipeline.yaml had some indentation inconsistency in one of the entries, however the error that came up was related to placeholders:
ploomber.exceptions.DAGSpecInitializationError: Failed to initialize spec. It looks like you're using placeholders (i.e. {{placeholder}}). Make sure values are enclosed in parentheses (e.g. key: "{{placeholder}}"). Original parser error:
mapping values are not allowed here
in "<unicode string>", line 14, column 11:
name: clean
do remember the pipeline.yaml contents so we can reproduce it?
Yes it was something like:
meta:
extract_product: False
tasks:
- source: raw.py
name: raw
product: {'nb': '{{path.products}}/data.ipynb', 'data': '{{path.products}}/data.csv'}
- source: clean_one.py
name: clean-1
product: {'nb': '{{path.products}}/clean_one.ipynb', 'data': '{{path.products}}/clean_one.csv'}
- source: clean_two.py
name: clean-2
product: { 'nb': '{{path.products}}/clean_two.ipynb', 'data': '{{path.products}}/clean_two.csv' }
- source: plot.py
name: plot
product: {'nb': '{{path.products}}/plot.ipynb'}
(Indentation is more in name and product keys of clean_one)
If we validate this on a yaml validator the error is (<unknown>): mapping values are not allowed in this context at line 10 column 10. So maybe if this error comes up we can throw error Possible indentation issue
Yeah, good point. So the idea with the "It looks like you're using placeholders..." was to provide a more user-friendly error message. But the scenario you encounter is a false positive. I think I'm only validating the error type and that the pipeline.yaml contains {{ and }}, but I guess we should also see if there's any other think we should validate before showing that message.
Regarding your suggestion, yes. I think the original "mapping values are not allowed..." message is confusing and we should add something like "you may have indentation issues"