clickhouse-mysql-data-reader icon indicating copy to clipboard operation
clickhouse-mysql-data-reader copied to clipboard

trying to import data from mysql but failing

Open aminbaig opened this issue 3 years ago • 2 comments

I have configured both mysql and clickhouse, when I try to import data I get the following error:

2021-04-27 12:08:01,107/1619507281.107731:DEBUG:class:<class 'clickhouse_mysql.writer.chwriter.CHWriter'> insert 1 event(s)
2021-04-27 12:08:01,481/1619507281.481600:DEBUG:class:<class 'clickhouse_mysql.writer.chwriter.CHWriter'> insert 100000 row(s)
2021-04-27 12:08:01,481/1619507281.481838:DEBUG:schema=vpbx table=incoming_calls self.dst_schema=None self.dst_table=None
2021-04-27 12:08:01,482/1619507281.482273:DEBUG:Connecting. Database: default. User: default
2021-04-27 12:08:01,482/1619507281.482419:DEBUG:Connecting to 127.0.0.1:9000
2021-04-27 12:08:01,484/1619507281.484395:DEBUG:Connected to ClickHouse server version 21.3.4, revision: 54447
2021-04-27 12:08:01,485/1619507281.485070:DEBUG:Query: INSERT INTO `vpbx`.`incoming_calls` (`id`, `client_id`, `date`, `time`, `cids`, `datetime`, `did_num`, `line_status`, `duration`, `billing_minutes`, `Last_App`, `Last_App_ID`, `Last_App_Data`, `call-status`, `dtmf`, `Caller_ID`, `Recd_File_Path`, `agent_name`, `agent_no`, `Lead_Check`) VALUES
2021-04-27 12:08:01,485/1619507281.485458:DEBUG:Block "" send time: 0.000073
2021-04-27 12:08:01,904/1619507281.904045:CRITICAL:QUERY FAILED
2021-04-27 12:08:01,904/1619507281.904250:CRITICAL:ex='NoneType' object has no attribute 'encode'
2021-04-27 12:08:01,904/1619507281.904330:CRITICAL:sql=INSERT INTO `vpbx`.`incoming_calls` (`id`, `client_id`, `date`, `time`, `cids`, `datetime`, `did_num`, `line_status`, `duration`, `billing_minutes`, `Last_App`, `Last_App_ID`, `Last_App_Data`, `call-status`, `dtmf`, `Caller_ID`, `Recd_File_Path`, `agent_name`, `agent_no`, `Lead_Check`) VALUES

Please advise.

aminbaig avatar Apr 27 '21 07:04 aminbaig

Try to add row[key] = '' if row[key] is None else row[key] in /usr/local/lib/python3.6/site-packages/clickhouse_mysql/writer/chwriter.py 77 line

a piece of code

           for row in event_converted:
                for key in row.keys():
                    row[key] = '' if row[key] is None else row[key]
                    # we need to convert Decimal value to str value for suitable for table structure
                    if type(row[key]) == Decimal:
                        row[key] = str(row[key])
                rows.append(row)

FessAectan avatar Oct 04 '21 13:10 FessAectan

I have encountered the same problem. How did you solve it

vgbhfive avatar Jan 04 '23 09:01 vgbhfive