snowflake-connector-python
snowflake-connector-python copied to clipboard
snowflakedb
SNOW-617164: SIGSEV / Segmentation Fault on `fetch_arrow_all` when pandas is not installed
Hey,
Currently, when I run the following code snippet
engine.connect().connection.cursor().execute(query_string).fetch_arrow_all()
a segmentation fault is raised when pandas is not installed. I know that pandas should be installed for .fetch_pandas_all() but I could not find an indication of it being needed for the pyarrow-functions as well.
Please answer these questions before submitting your issue. Thanks!
-
What version of Python are you using? Python 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:03:09) [Clang 13.0.1 ]
-
What operating system and processor architecture are you using? macOS-12.3-x86_64-i386-64bit
-
What are the component versions in the environment (
pip freeze)?
abseil-cpp 20210324.2 he49afe7_0 conda-forge
alabaster 0.7.12 py_0 conda-forge
arrow-cpp 7.0.0 py310hf221cfc_8_cpu conda-forge
asn1crypto 1.5.1 pyhd8ed1ab_0 conda-forge
attrs 21.4.0 pyhd8ed1ab_0 conda-forge
aws-c-cal 0.5.11 hd2e2f4b_0 conda-forge
aws-c-common 0.6.2 h0d85af4_0 conda-forge
aws-c-event-stream 0.2.7 hb9330a7_13 conda-forge
aws-c-io 0.10.5 h35aa462_0 conda-forge
aws-checksums 0.1.11 h0010a65_7 conda-forge
aws-sdk-cpp 1.8.186 h766a74d_3 conda-forge
babel 2.10.3 pyhd8ed1ab_0 conda-forge
brotlipy 0.7.0 py310h1961e1f_1004 conda-forge
bzip2 1.0.8 h0d85af4_4 conda-forge
c-ares 1.18.1 h0d85af4_0 conda-forge
ca-certificates 2022.6.15 h033912b_0 conda-forge
certifi 2022.6.15 py310h2ec42d9_0 conda-forge
cffi 1.15.0 py310hcc37b68_0 conda-forge
cfgv 3.3.1 pyhd8ed1ab_0 conda-forge
charset-normalizer 2.0.12 pyhd8ed1ab_0 conda-forge
colorama 0.4.5 pyhd8ed1ab_0 conda-forge
conda 4.13.0 py310h2ec42d9_1 conda-forge
conda-package-handling 1.8.1 py310h1961e1f_1 conda-forge
conda-tree 1.0.5 pyhd8ed1ab_0 conda-forge
coverage 6.4.1 py310h6c45266_0 conda-forge
cryptography 35.0.0 py310ha82f1d4_2 conda-forge
distlib 0.3.4 pyhd8ed1ab_0 conda-forge
docutils 0.17.1 py310h2ec42d9_2 conda-forge
filelock 3.7.1 pyhd8ed1ab_0 conda-forge
flit 3.7.1 pyhd8ed1ab_0 conda-forge
flit-core 3.7.1 pyhd8ed1ab_0 conda-forge
freetds 1.1.15 h6f4450b_0 conda-forge
gflags 2.2.2 hb1e8313_1004 conda-forge
glog 0.6.0 h8ac2a54_0 conda-forge
gmp 6.2.1 h2e338ed_0 conda-forge
greenlet 1.1.2 py310h9d931ec_2 conda-forge
grpc-cpp 1.45.2 hb472a99_3 conda-forge
identify 2.5.1 pyhd8ed1ab_0 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
imagesize 1.3.0 pyhd8ed1ab_0 conda-forge
importlib-metadata 4.11.4 py310h2ec42d9_0 conda-forge
iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge
jinja2 3.0.3 pyhd8ed1ab_0 conda-forge
krb5 1.19.3 hb49756b_0 conda-forge
libblas 3.9.0 15_osx64_openblas conda-forge
libbrotlicommon 1.0.9 h5eb16cf_7 conda-forge
libbrotlidec 1.0.9 h5eb16cf_7 conda-forge
libbrotlienc 1.0.9 h5eb16cf_7 conda-forge
libcblas 3.9.0 15_osx64_openblas conda-forge
libcrc32c 1.1.2 he49afe7_0 conda-forge
libcurl 7.83.1 h372c54d_0 conda-forge
libcxx 14.0.5 hce7ea42_1 conda-forge
libedit 3.1.20191231 h0678c8f_2 conda-forge
libev 4.33 haf1e3a3_1 conda-forge
libevent 2.1.10 h815e4d9_4 conda-forge
libffi 3.4.2 h0d85af4_5 conda-forge
libgfortran 5.0.0 9_3_0_h6c81a4c_23 conda-forge
libgfortran5 9.3.0 h6c81a4c_23 conda-forge
libgoogle-cloud 1.40.2 hc6327d4_0 conda-forge
libiconv 1.16 haf1e3a3_0 conda-forge
liblapack 3.9.0 15_osx64_openblas conda-forge
libnghttp2 1.47.0 h942079c_0 conda-forge
libopenblas 0.3.20 openmp_hb3cd9ec_0 conda-forge
libpq 14.4 hf6bb32a_0 conda-forge
libprotobuf 3.20.1 h2292cb8_0 conda-forge
libssh2 1.10.0 h52ee1ee_2 conda-forge
libthrift 0.16.0 h9702cd6_1 conda-forge
libutf8proc 2.7.0 h0d85af4_0 conda-forge
libzlib 1.2.12 hfe4f2af_1 conda-forge
llvm-openmp 14.0.4 ha654fa7_0 conda-forge
lz4-c 1.9.3 he49afe7_1 conda-forge
make 4.3 h22f3db7_1 conda-forge
markupsafe 2.1.1 py310h1961e1f_1 conda-forge
ncurses 6.3 h96cf925_1 conda-forge
networkx 2.8.4 pyhd8ed1ab_0 conda-forge
nodeenv 1.6.0 pyhd8ed1ab_0 conda-forge
numpy 1.22.4 py310hed37afb_0 conda-forge
numpydoc 1.4.0 pyhd8ed1ab_0 conda-forge
openssl 1.1.1p hfe4f2af_0 conda-forge
orc 1.7.5 h4856350_0 conda-forge
oscrypto 1.2.1 pyhd3deb0d_0 conda-forge
packaging 21.3 pyhd8ed1ab_0 conda-forge
parquet-cpp 1.5.1 1 conda-forge
pbr 5.9.0 pyhd8ed1ab_0 conda-forge
pip 22.1.2 pyhd8ed1ab_0 conda-forge
platformdirs 2.5.1 pyhd8ed1ab_0 conda-forge
pluggy 1.0.0 py310h2ec42d9_3 conda-forge
pre-commit 2.19.0 py310h2ec42d9_0 conda-forge
psycopg2 2.9.3 py310hff56b6c_0 conda-forge
py 1.11.0 pyh6c4a22f_0 conda-forge
pyarrow 7.0.0 py310hb135652_8_cpu conda-forge
pycosat 0.6.3 py310h1961e1f_1010 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pycryptodomex 3.14.1 py310h1961e1f_1 conda-forge
pygments 2.12.0 pyhd8ed1ab_0 conda-forge
pyjwt 2.4.0 pyhd8ed1ab_0 conda-forge
pyodbc 4.0.32 py310hba3363e_1 conda-forge
pyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 py310h2ec42d9_5 conda-forge
pytest 7.1.2 py310h2ec42d9_0 conda-forge
pytest-cov 3.0.0 pyhd8ed1ab_0 conda-forge
pytest-html 3.1.1 pyhd8ed1ab_0 conda-forge
pytest-metadata 2.0.1 pyhd8ed1ab_0 conda-forge
python 3.10.5 hdaaf3db_0_cpython conda-forge
python_abi 3.10 2_cp310 conda-forge
pytz 2022.1 pyhd8ed1ab_0 conda-forge
pyyaml 6.0 py310h1961e1f_4 conda-forge
re2 2022.04.01 h96cf925_0 conda-forge
readline 8.1.2 h3899abd_0 conda-forge
requests 2.28.0 pyhd8ed1ab_0 conda-forge
ruamel_yaml 0.15.80 py310h6c45266_1007 conda-forge
scipy 1.8.1 py310h1f9c157_0 conda-forge
setuptools 62.6.0 py310h2ec42d9_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.9 h6e38e02_1 conda-forge
snowballstemmer 2.2.0 pyhd8ed1ab_0 conda-forge
snowflake-connector-python 2.7.8 py310h692597a_0 conda-forge
snowflake-sqlalchemy 1.3.4 pyhd8ed1ab_0 conda-forge
sphinx 5.0.2 pyh6c4a22f_0 conda-forge
sphinx_rtd_theme 1.0.0 pyhd8ed1ab_0 conda-forge
sphinxcontrib-apidoc 0.3.0 py_1 conda-forge
sphinxcontrib-applehelp 1.0.2 py_0 conda-forge
sphinxcontrib-devhelp 1.0.2 py_0 conda-forge
sphinxcontrib-htmlhelp 2.0.0 pyhd8ed1ab_0 conda-forge
sphinxcontrib-jsmath 1.0.1 py_0 conda-forge
sphinxcontrib-qthelp 1.0.3 py_0 conda-forge
sphinxcontrib-serializinghtml 1.1.5 pyhd8ed1ab_2 conda-forge
sqlalchemy 1.4.37 py310h6c45266_0 conda-forge
sqlite 3.38.5 hd9f0692_0 conda-forge
tk 8.6.12 h5dbffcc_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
tomli-w 1.0.0 pyhd8ed1ab_0 conda-forge
tqdm 4.64.0 pyhd8ed1ab_0 conda-forge
tzdata 2022a h191b570_0 conda-forge
ukkonen 1.0.1 py310h6be76da_2 conda-forge
unixodbc 2.3.10 h7b58acd_0 conda-forge
urllib3 1.26.9 pyhd8ed1ab_0 conda-forge
virtualenv 20.14.1 py310h2ec42d9_0 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
xz 5.2.5 haf1e3a3_1 conda-forge
yaml 0.2.5 h0d85af4_2 conda-forge
zipp 3.8.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.12 hfe4f2af_1 conda-forge
zstd 1.5.2 ha9df2e0_1 conda-forge
-
What did you do?
When calling
.fetch_arrow_all()without havingpandasinstalled while havingpyarrowinstalled, a segmentation fault is raised with the following output
test_integration.py::test_ks_2sample_constraint_perfect_between[snowflake-data0] Fatal Python error: Segmentation fault
Current thread 0x0000000108865600 (most recent call first):
File "/usr/local/Caskroom/miniconda/base/envs/datajudge/lib/python3.10/site-packages/snowflake/connector/result_set.py", line 68 in result_set_iterator
File "/usr/local/Caskroom/miniconda/base/envs/datajudge/lib/python3.10/site-packages/snowflake/connector/result_set.py", line 168 in _fetch_arrow_all
File "/usr/local/Caskroom/miniconda/base/envs/datajudge/lib/python3.10/site-packages/snowflake/connector/cursor.py", line 961 in fetch_arrow_all
# user-specific code
Extension modules: sqlalchemy.cimmutabledict, greenlet._greenlet, sqlalchemy.cprocessors, sqlalchemy.cresultproxy, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, _cffi_backend, snowflake.connector.arrow_iterator (total: 19)
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
The final segmentation fault, therefore, seems to occur here.
-
What did you expect to see? I expected the function to return the
pyarrow-Table regularly. I am currently not aware of how the code should depend onpandasat any point. Installing justpyarrowshould be enough. (Though I'm not too familiar with the snowflake-connector codebase). -
Can you set logging to DEBUG and collect the logs? log.txt
To clean up and re-prioritize bugs and feature requests we are closing all issues older than 6 months as of March 1, 2023. If there are any issues or feature requests that you would like us to address, please re-create them. For urgent issues, opening a support case with this link Snowflake Community is the fastest way to get a response
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}