jitsu
jitsu copied to clipboard
Improve patch table function
Problem
Currently patchTable checks difference between table schema in destination and table schema derived from data. But it ignores column type difference completely. So if column types differ we attempt to do insert and fail with error that not always conclusive.
Solution
- tableHelper.EnsureTable: must return table schema with column types derived from current data (right now types comes from destination table)
- tableHelper.EnsureTable: must additionally return column types diffs object
- if consequent insert fails we need to print extra error about column types diffs that could cause this error and offer solution: sql_type mapping or alter table
Optional:
- For sources sync only we can find common type parents for conflicting columns, create new table with fixed types and fill it with data from destination_table and tmp_table