sayn icon indicating copy to clipboard operation
sayn copied to clipboard

Copy task fails for tables with `json` columns

Open tonbadal opened this issue 3 years ago • 0 comments

I'm running a copy task with a table users that contains a metadata column with type json.

tasks:
  copy_users:  
    type: copy
    source:
      db: input_db
      table: users
    destination:
      db: output_db
      table: users

And when running the pipeline I get the following error:

INFO|Run Steps: Prepare Load, Load Data, Move Table
INFO|[1/3]  Executing Prepare Load
INFO|[1/3]  Prepare Load (97.8ms)
INFO|[2/3]  Executing Load Data
ERROR|Failed (2s) invalid input syntax for type json
DETAIL:  Token "'" is invalid.
CONTEXT:  JSON data, line 1: {'...
COPY sayn_tmp_users, line 10, column metadata: "{'title': 'Engineer'}"

I've successfully copied other tables that don't have json columns.

Both input and output databases are PostgreSQL, runnning in Python 3.9.13, SAYN version 0.6.5.

Many thanks in advance!

tonbadal avatar Jul 30 '22 17:07 tonbadal