hooli-data-eng-pipelines
hooli-data-eng-pipelines copied to clipboard
use stream definition for group
Small change, but useful to show how you could use the stream_definition to determine the asset group name (similar to how we do so with dbt metadata
format of stream_definition
looks like:
# stream_definition example
# {'name': 's3://hooli-demo/embedded-elt/locations.csv', 'config': {'object': 'locations'}}