connector-x
connector-x copied to clipboard
python connectorx query database data fail "Packets out of sync"
What language are you using?
Python
What version are you using?
connectorx=0.3.1
What database are you using?
I use starrocks-3.2.4 database.
What dataframe are you using?
PyMySQL=1.0.2
pyarrow=8.0.0
pandas=1.5.0
numpy=1.22.4
Can you describe your bug?
I can connect to database and query for data base on pymysql.
However when i use connectorx to do same query, it fail.
- When i query using sql
select * from table_name
, get errorCouldn't convert Row {id: Bytes("1"), state: Bytes("Success")} to type alloc::string::String.
- When i query using
select state from table_name
, get errorCodeError {Packets out of sync}
Besides, i run with setting env variable RUST_BACKTRACE=full
, get the detail error message below
thread '<unnamed>' panicked at 'Couldn't convert Row { id: Bytes("23"), state: Bytes("Success") } to type alloc::string::String. (see FromRow documentation)', /github/home/.cargo/registry/src/github.com-1ecc6299db9ec823/mysql_common-0.27.5/src/row/convert/mod.rs:81:39
stack backtrace:
0: rust_begin_unwind
at /rustc/750bd1a7ff3e010611b97ee75d30b7cbf5f3a03c/library/std/src/panicking.rs:584:5
1: core::panicking::panic_fmt
at /rustc/750bd1a7ff3e010611b97ee75d30b7cbf5f3a03c/library/core/src/panicking.rs:142:14
2: mysql_common::row::convert::FromRow::from_row
3: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
4: mysql::conn::queryable::Queryable::query
5: <r2d2_mysql::pool::MysqlConnectionManager as r2d2::ManageConnection>::has_broken
6: <r2d2::PooledConnection<M> as core::ops::drop::Drop>::drop
7: core::ptr::drop_in_place<r2d2::PooledConnection<r2d2_mysql::pool::MysqlConnectionManager>>
8: <connectorx::sources::mysql::MySQLSource<P> as connectorx::sources::Source>::fetch_metadata
9: connectorx::dispatcher::Dispatcher<S,D,TP>::run
10: connectorx::get_arrow::get_arrow
11: connectorx::arrow::write_arrow
12: connectorx::read_sql
13: std::panicking::try
14: connectorx::__pyo3_raw_read_sql
15: cfunction_vectorcall_FASTCALL_KEYWORDS
at /usr/local/src/conda/python-3.8.16/Objects/methodobject.c:441:24
16: _PyObject_Vectorcall
at /usr/local/src/conda/python-3.8.16/Include/cpython/abstract.h:127:11
17: call_function
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:4963:13
18: _PyEval_EvalFrameDefault
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:3515:19
19: PyEval_EvalFrameEx
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:741:12
20: _PyEval_EvalCodeWithName
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:4298:14
21: _PyFunction_Vectorcall
at /usr/local/src/conda/python-3.8.16/Objects/call.c:436:12
22: _PyObject_Vectorcall
at /usr/local/src/conda/python-3.8.16/Include/cpython/abstract.h:127:11
23: call_function
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:4963:13
24: _PyEval_EvalFrameDefault
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:3515:19
25: PyEval_EvalFrameEx
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:741:12
26: _PyEval_EvalCodeWithName
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:4298:14
27: PyEval_EvalCodeEx
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:4327:12
28: PyEval_EvalCode
at /usr/local/src/conda/python-3.8.16/Python/ceval.c:718:12
29: run_eval_code_obj
at /usr/local/src/conda/python-3.8.16/Python/pythonrun.c:1166:9
30: run_mod
at /usr/local/src/conda/python-3.8.16/Python/pythonrun.c:1188:9
31: pyrun_file
at /usr/local/src/conda/python-3.8.16/Python/pythonrun.c:1085:15
32: pyrun_simple_file
at /usr/local/src/conda/python-3.8.16/Python/pythonrun.c:439:13
33: PyRun_SimpleFileExFlags
at /usr/local/src/conda/python-3.8.16/Python/pythonrun.c:472:15
34: pymain_run_file
at /usr/local/src/conda/python-3.8.16/Modules/main.c:391:15
35: pymain_run_python
at /usr/local/src/conda/python-3.8.16/Modules/main.c:616:21
36: Py_RunMain
at /usr/local/src/conda/python-3.8.16/Modules/main.c:695:5
37: Py_BytesMain
at /usr/local/src/conda/python-3.8.16/Modules/main.c:1127:12
38: __libc_start_main
39: <unknown>
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Traceback (most recent call last):
File "connectorx_debug.py", line 38, in <module>
df = cx.read_sql(aiyield_engine_url, sql2, protocol='text', return_type='arrow')
File "/home/amedac/miniconda3/envs/etl_env/lib/python3.8/site-packages/connectorx/__init__.py", line 257, in read_sql
result = _read_sql(
It seem like some system basic lib is incompatible, could you give me some hint to
What are the steps to reproduce the behavior?
sample code:
import connectorx as cx
sql1 = 'select * from table_name'
sql2 = 'select state from table_name'
url = f'mysql://{user_name}:{passwd}@{host}:{port}/{db_name}'
df = cx.read_sql(url, sql1, protocol='text', return_type='arrow')
Database setup if the error only happens on specific data or data type
field_name | type |
---|---|
id | INT |
state | string |