hooli-data-eng-pipelines icon indicating copy to clipboard operation
hooli-data-eng-pipelines copied to clipboard

Published 20 hours ago • verified publisher icon dagster-io

Update sling assets

Open izzye84 opened this issue 9 months ago • 3 comments

Updating the Sling demo to the new APIs.

Summary of updates:

  • added replication python config
  • updated SlingResource to use SlingConnectionResource instead of SlingSourceConnection/SlingTargetConnection (soon to be deprecated)
  • add custom sling translator to modify the default asset key (prefix with "RAW_DATA")
  • updated sling assets to use the sling_assets decorator

There seems to be a bug with local dev when including a target schema with duckdb — "Catalog Error: Schema with name "raw_data" already exists!"

duck_db_target_schema_error

izzye84 avatar May 17 '24 00:05 izzye84