dbt-databricks icon indicating copy to clipboard operation
dbt-databricks copied to clipboard

Snapshotting silently fails on values greater than int max value when source dtype changes to bigint

Open jelstongreen opened this issue 1 year ago • 0 comments
trafficstars

Describe the bug

We noticed one table that we snapshot that when the source data changes a col from int to bigint but the downstream snapshot is not amended that the snapshot will just continuously insert the old record whenever the snapshot is run and not error that the new value cannot be inserted.

Steps To Reproduce

Snapshot a source model with a regular int column. Change the source model column to bigint and change the value in the row to a bigint value larger than max int. Run the dbt snapshot on the source table. The snapshot should have the check strategy and have the column that's been amended in that check list.

Expected behavior

A clear and concise description of what you expected to happen. If the target dtype is int for a bigint value then there should be an error rather than a silent failure.

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

System information

The output of dbt --version:

<output goes here>

The operating system you're using:

The output of python --version:

Additional context

Add any other context about the problem here.

jelstongreen avatar Aug 07 '24 11:08 jelstongreen