ploomber icon indicating copy to clipboard operation
ploomber copied to clipboard

Error when pipeline.yaml file has indentation issue

Open neelasha23 opened this issue 3 years ago • 3 comments

Pipeline.yaml had some indentation inconsistency in one of the entries, however the error that came up was related to placeholders:

ploomber.exceptions.DAGSpecInitializationError: Failed to initialize spec. It looks like you're using placeholders (i.e. {{placeholder}}). Make sure values are enclosed in parentheses (e.g. key: "{{placeholder}}"). Original parser error:
                  
                  mapping values are not allowed here
                     in "<unicode string>", line 14, column 11:
                             name: clean

neelasha23 avatar Jun 18 '22 16:06 neelasha23

do remember the pipeline.yaml contents so we can reproduce it?

edublancas avatar Jun 20 '22 20:06 edublancas

Yes it was something like:

meta:
  extract_product: False

tasks:
  - source: raw.py
    name: raw
    product: {'nb': '{{path.products}}/data.ipynb', 'data': '{{path.products}}/data.csv'}

  - source: clean_one.py
     name: clean-1
     product: {'nb': '{{path.products}}/clean_one.ipynb', 'data': '{{path.products}}/clean_one.csv'}

  - source: clean_two.py
    name: clean-2
    product: { 'nb': '{{path.products}}/clean_two.ipynb', 'data': '{{path.products}}/clean_two.csv' }

  - source: plot.py
    name: plot
    product: {'nb': '{{path.products}}/plot.ipynb'}

(Indentation is more in name and product keys of clean_one)

If we validate this on a yaml validator the error is (<unknown>): mapping values are not allowed in this context at line 10 column 10. So maybe if this error comes up we can throw error Possible indentation issue

neelasha23 avatar Jun 21 '22 03:06 neelasha23

Yeah, good point. So the idea with the "It looks like you're using placeholders..." was to provide a more user-friendly error message. But the scenario you encounter is a false positive. I think I'm only validating the error type and that the pipeline.yaml contains {{ and }}, but I guess we should also see if there's any other think we should validate before showing that message.

Regarding your suggestion, yes. I think the original "mapping values are not allowed..." message is confusing and we should add something like "you may have indentation issues"

edublancas avatar Jun 21 '22 06:06 edublancas