etl
etl copied to clipboard
Remove redundant grapher steps
Most of our grapher steps are just propagating the same dataset to the grapher channel. It's not a big deal, but it's one more dataset to store and maintain. We had some ideas for simplification:
- Instead of
grapher://grapher/namespace/version/dataset:
- data://grapher/namespace/version/dataset
data://grapher/namespace/version/dataset
- data://garden/namespace/version/dataset
simply build grapher from garden dataset
grapher://grapher/namespace/version/dataset:
- data://garden/namespace/version/dataset
- If there's a YAML file in step directory without
*.py
file, take the garden dataset and enhance it with the YAML file