hooli-data-eng-pipelines icon indicating copy to clipboard operation
hooli-data-eng-pipelines copied to clipboard

use stream definition for group

Open cnolanminich opened this issue 7 months ago • 1 comments

Small change, but useful to show how you could use the stream_definition to determine the asset group name (similar to how we do so with dbt metadata

format of stream_definition looks like:

      # stream_definition example 
      # {'name': 's3://hooli-demo/embedded-elt/locations.csv', 'config': {'object': 'locations'}}

cnolanminich avatar Jul 29 '24 16:07 cnolanminich