datapackage-pipelines icon indicating copy to clipboard operation
datapackage-pipelines copied to clipboard

dump.to_sql should detect schema changes

Open OriHoch opened this issue 7 years ago • 0 comments

reproduction steps

  • use dump.to_sql with a field defined as date
  • when dump.to_sql is run it will create the table and defined the column as a date type in the sql engine
  • modify the code, now the field is defined as string
  • re-run the pipeline

expected

  • dump.to_sql should fail and warn that the schema changed and now the descriptor schema doesn't match the sql schema

actual

  • dump.to_sql tries to update the field (assuming it's in the correct type) and fails, showing a hard to debug error message

notes

  • the jsontableschema_sql library has a method that returns the descriptor from an sql table - if it works it should provide the needed functionality

OriHoch avatar Jun 11 '17 08:06 OriHoch