sling-cli icon indicating copy to clipboard operation
sling-cli copied to clipboard

Empty strings are treated as null when syncing from Snowflake to Postgres, causing failures on non-null constraints

Open nicklausroach opened this issue 1 year ago • 1 comments

Issue Description

  • Description of the issue: After upgrading to v1.2.14, our syncs from Snowflake to Postgres are failing. This seems to be due to empty strings in a field being treated as nulls, resulting in a failure when hitting our non-null constraints.

  • Sling version (sling --version): 1.2.14

  • Operating System (linux, mac, windows): linux

  • Replication Configuration: Generated by custom Dagster resource

  • Log Output (please run command with -d):

fatal:
~ Error executing: insert into "public"."usage_metrics_daily_jobs_aggregated" ("organization_id", "agent_type", "job_name", "repository_name", "job_day", "materializations", "step_duration_mins", "steps", "run_duration_mins", "runs", "steps_credits") select "organization_id", "agent_type", "job_name", "repository_name", "job_day"::date as "job_day", "materializations", "step_duration_mins", "steps", "run_duration_mins", "runs", "steps_credits" from "public"."usage_metrics_daily_jobs_aggregated_tmp"
pq: null value in column "repository_name" of relation "usage_metrics_daily_jobs_aggregated" violates not-null constraint

nicklausroach avatar Jul 30 '24 02:07 nicklausroach

Thanks for reporting.

flarco avatar Jul 30 '24 10:07 flarco

Should be good now with https://github.com/slingdata-io/sling-cli/pull/344/commits/9644af3ed432a382030418fed9111aff50bbf01c Closing.

flarco avatar Aug 14 '24 00:08 flarco