clickhouse-mysql-data-reader
clickhouse-mysql-data-reader copied to clipboard
trying to import data from mysql but failing
I have configured both mysql and clickhouse, when I try to import data I get the following error:
2021-04-27 12:08:01,107/1619507281.107731:DEBUG:class:<class 'clickhouse_mysql.writer.chwriter.CHWriter'> insert 1 event(s)
2021-04-27 12:08:01,481/1619507281.481600:DEBUG:class:<class 'clickhouse_mysql.writer.chwriter.CHWriter'> insert 100000 row(s)
2021-04-27 12:08:01,481/1619507281.481838:DEBUG:schema=vpbx table=incoming_calls self.dst_schema=None self.dst_table=None
2021-04-27 12:08:01,482/1619507281.482273:DEBUG:Connecting. Database: default. User: default
2021-04-27 12:08:01,482/1619507281.482419:DEBUG:Connecting to 127.0.0.1:9000
2021-04-27 12:08:01,484/1619507281.484395:DEBUG:Connected to ClickHouse server version 21.3.4, revision: 54447
2021-04-27 12:08:01,485/1619507281.485070:DEBUG:Query: INSERT INTO `vpbx`.`incoming_calls` (`id`, `client_id`, `date`, `time`, `cids`, `datetime`, `did_num`, `line_status`, `duration`, `billing_minutes`, `Last_App`, `Last_App_ID`, `Last_App_Data`, `call-status`, `dtmf`, `Caller_ID`, `Recd_File_Path`, `agent_name`, `agent_no`, `Lead_Check`) VALUES
2021-04-27 12:08:01,485/1619507281.485458:DEBUG:Block "" send time: 0.000073
2021-04-27 12:08:01,904/1619507281.904045:CRITICAL:QUERY FAILED
2021-04-27 12:08:01,904/1619507281.904250:CRITICAL:ex='NoneType' object has no attribute 'encode'
2021-04-27 12:08:01,904/1619507281.904330:CRITICAL:sql=INSERT INTO `vpbx`.`incoming_calls` (`id`, `client_id`, `date`, `time`, `cids`, `datetime`, `did_num`, `line_status`, `duration`, `billing_minutes`, `Last_App`, `Last_App_ID`, `Last_App_Data`, `call-status`, `dtmf`, `Caller_ID`, `Recd_File_Path`, `agent_name`, `agent_no`, `Lead_Check`) VALUES
Please advise.
Try to add row[key] = '' if row[key] is None else row[key] in /usr/local/lib/python3.6/site-packages/clickhouse_mysql/writer/chwriter.py 77 line
a piece of code
for row in event_converted:
for key in row.keys():
row[key] = '' if row[key] is None else row[key]
# we need to convert Decimal value to str value for suitable for table structure
if type(row[key]) == Decimal:
row[key] = str(row[key])
rows.append(row)
I have encountered the same problem. How did you solve it