snowflake-connector-python icon indicating copy to clipboard operation
snowflake-connector-python copied to clipboard

Published 20 hours ago • verified publisher icon snowflakedb

SNOW-588043: Method of reading number from SQL can return int8 or int64

Open DamianBarabonkovQC opened this issue 2 years ago • 5 comments

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using? Python 3.10.4

  2. What operating system and processor architecture are you using? macOS-11.6-x86_64-i386-64bit

  3. What are the component versions in the environment (pip freeze)? SQLAlchemy @ file:///Users/runner/miniforge3/conda-bld/sqlalchemy_1651018002635/work snowflake-connector-python @ file:///Users/runner/miniforge3/conda-bld/snowflake-connector-python_1647699769100/work snowflake-sqlalchemy @ file:///home/conda/feedstock_root/build_artifacts/snowflake-sqlalchemy_1651130030626/work

  4. What did you do?

import os
import pandas as pd
import snowflake.connector

account = '****'
user = ''****''
password = ''****''
role = ''****''
warehouse = ''****''
database = ''****''
schema = ''****''
table_name = ''****''

credentials = {
    "account": account,
    "user": user,
    "password": password,
    "role": role,
    "warehouse": warehouse,
    "database": database,
    "schema": schema,
}

conn = snowflake.connector.connect(**credentials)
cur = conn.cursor()

cur.execute(f'CREATE or REPLACE TABLE {database}.{schema}.{table_name} (CHUNK BIGINT)')
cur.execute(f'INSERT INTO {database}.{schema}.{table_name} VALUES (0)')
cur.execute(f'INSERT INTO {database}.{schema}.{table_name} VALUES (1)')
cur.execute(f'INSERT INTO {database}.{schema}.{table_name} VALUES (2)')
cur.execute(f'INSERT INTO {database}.{schema}.{table_name} VALUES (3)')
cur.execute(f'INSERT INTO {database}.{schema}.{table_name} VALUES (4)')


def fetch_pandas_returns_int64(cur, sql):
    cur.execute(sql)
    dat = cur.fetchmany(50000)
    return pd.DataFrame(dat, columns=cur.description)

def fetch_pandas_returns_int8(cur, sql):
    cur.execute(sql)
    batches = cur.get_result_batches()
    return batches[0].to_pandas()

# Desired behavior, the column was created as a 64bit BIGINT
print(df := fetch_pandas_returns_int64(cur, f'SELECT CHUNK FROM {schema}.{table_name}'))
print(df.dtypes)

# Undesired behavior, the column was created as a 64bit BIGINT
print(df := fetch_pandas_returns_int8(cur, f'SELECT CHUNK FROM {schema}.{table_name}'))
print(df.dtypes)

cur.close()
conn.close()
  1. What did you expect to see?

The table was created with a column named CHUNK with the type BIGINT. I expect that reading this value into a pandas dataframe would produce an int64 type. However, when it is read in batches, the type is inferred by the values. Since I populated the column with small values, the type is an int8. If the values are a little larger, it is read as an int16.

  1. Can you set logging to DEBUG and collect the logs?
Click to expand 
2022-05-10 16:46:36,608 - MainThread connection.py:257 - __init__() - INFO - Snowflake Connector for Python Version: 2.7.6, Python Version: 3.10.4, Platform: macOS-11.6-x86_64-i386-64bit
2022-05-10 16:46:36,609 - MainThread connection.py:489 - connect() - DEBUG - connect
2022-05-10 16:46:36,609 - MainThread connection.py:754 - __config() - DEBUG - __config
2022-05-10 16:46:36,609 - MainThread connection.py:878 - __config() - INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
2022-05-10 16:46:36,609 - MainThread connection.py:896 - __config() - INFO - Setting use_openssl_only mode to False
2022-05-10 16:46:36,609 - MainThread converter.py:143 - __init__() - DEBUG - use_numpy: False
2022-05-10 16:46:36,609 - MainThread connection.py:657 - __open_connection() - DEBUG - REST API object was created: ag44040.europe-west4.gcp.snowflakecomputing.com:443
2022-05-10 16:46:36,609 - MainThread auth.py:170 - authenticate() - DEBUG - authenticate
2022-05-10 16:46:36,609 - MainThread auth.py:200 - authenticate() - DEBUG - assertion content: *********
2022-05-10 16:46:36,609 - MainThread auth.py:203 - authenticate() - DEBUG - account=ag44040, user=dbarabonkov, database=QUANTCORE_THEK, schema=PUBLIC, warehouse=THEK, role=ACCOUNTADMIN, request_id=a7f15ae3-9caa-4f3f-bc14-5ee960812e11
2022-05-10 16:46:36,609 - MainThread auth.py:236 - authenticate() - DEBUG - body['data']: {'CLIENT_APP_ID': 'PythonConnector', 'CLIENT_APP_VERSION': '2.7.6', 'SVN_REVISION': None, 'ACCOUNT_NAME': 'ag44040', 'LOGIN_NAME': 'dbarabonkov', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'PythonConnector', 'OS': 'Darwin', 'OS_VERSION': 'macOS-11.6-x86_64-i386-64bit', 'PYTHON_VERSION': '3.10.4', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'Clang 12.0.1 ', 'OCSP_MODE': 'FAIL_OPEN', 'TRACING': 10, 'LOGIN_TIMEOUT': 120, 'NETWORK_TIMEOUT': None}, 'SESSION_PARAMETERS': {'CLIENT_PREFETCH_THREADS': 4}}
2022-05-10 16:46:36,609 - MainThread auth.py:254 - authenticate() - DEBUG - Timeout set to 120
2022-05-10 16:46:36,610 - MainThread retry.py:333 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2022-05-10 16:46:36,610 - MainThread retry.py:333 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2022-05-10 16:46:36,610 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:36,610 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: 120, retry cnt: 1
2022-05-10 16:46:36,610 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: bf7ef91d-4119-412a-ad67-f367e7195523
2022-05-10 16:46:36,610 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:36,689 - MainThread connectionpool.py:971 - _new_conn() - DEBUG - Starting new HTTPS connection (1): ag44040.europe-west4.gcp.snowflakecomputing.com:443
2022-05-10 16:46:36,801 - MainThread ocsp_snowflake.py:465 - reset_cache_dir() - DEBUG - cache directory: /Users/damianbarabonkov/Library/Caches/Snowflake
2022-05-10 16:46:36,875 - MainThread ssl_wrap_socket.py:80 - ssl_wrap_socket_with_ocsp() - DEBUG - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
2022-05-10 16:46:36,875 - MainThread ocsp_snowflake.py:507 - reset_ocsp_response_cache_uri() - DEBUG - ocsp_response_cache_uri: file:///Users/damianbarabonkov/Library/Caches/Snowflake/ocsp_response_cache.json
2022-05-10 16:46:36,875 - MainThread ocsp_snowflake.py:508 - reset_ocsp_response_cache_uri() - DEBUG - OCSP_VALIDATION_CACHE size: 0
2022-05-10 16:46:36,875 - MainThread ocsp_snowflake.py:290 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP response cache server is enabled: http://ocsp.snowflakecomputing.com/ocsp_response_cache.json
2022-05-10 16:46:36,875 - MainThread ocsp_snowflake.py:326 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP dynamic cache server RETRY URL: None
2022-05-10 16:46:36,878 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2022-05-10 16:46:36,879 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2022-05-10 16:46:36,880 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2022-05-10 16:46:36,899 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-05-12 02:43:45+00:00
2022-05-10 16:46:36,900 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-05-12 02:47:38+00:00
2022-05-10 16:46:36,901 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2022-07-15 00:00:00+00:00
2022-05-10 16:46:36,902 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,902 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,903 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,904 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,904 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 20:00:20+00:00
2022-05-10 16:46:36,905 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,906 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 20:00:20+00:00
2022-05-10 16:46:36,907 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,907 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 20:00:20+00:00
2022-05-10 16:46:36,908 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,909 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,909 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,910 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 20:00:20+00:00
2022-05-10 16:46:36,911 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,912 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,912 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,913 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 20:00:20+00:00
2022-05-10 16:46:36,914 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,915 - MainThread ocsp_asn1crypto.py:234 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-01-25 19:35:21+00:00
2022-05-10 16:46:36,916 - MainThread ocsp_snowflake.py:543 - read_ocsp_response_cache_file() - DEBUG - Read OCSP response cache file: /Users/damianbarabonkov/Library/Caches/Snowflake/ocsp_response_cache.json, count=175
2022-05-10 16:46:36,916 - MainThread ocsp_snowflake.py:1063 - validate() - DEBUG - validating certificate: ag44040.europe-west4.gcp.snowflakecomputing.com
2022-05-10 16:46:36,916 - MainThread ocsp_asn1crypto.py:435 - extract_certificate_chain() - DEBUG - # of certificates: 3
2022-05-10 16:46:36,917 - MainThread ocsp_asn1crypto.py:440 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'California'), ('locality_name', 'San Mateo'), ('organization_name', 'Snowflake Inc.'), ('common_name', '*.europe-west4.gcp.snowflakecomputing.com')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert TLS RSA SHA256 2020 CA1')])
2022-05-10 16:46:36,917 - MainThread ocsp_asn1crypto.py:440 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert TLS RSA SHA256 2020 CA1')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root CA')])
2022-05-10 16:46:36,917 - MainThread ocsp_asn1crypto.py:440 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root CA')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root CA')])
2022-05-10 16:46:36,919 - MainThread ocsp_snowflake.py:683 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'California'), ('locality_name', 'San Mateo'), ('organization_name', 'Snowflake Inc.'), ('common_name', '*.europe-west4.gcp.snowflakecomputing.com')])
2022-05-10 16:46:36,919 - MainThread ocsp_snowflake.py:683 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert TLS RSA SHA256 2020 CA1')])
2022-05-10 16:46:36,920 - MainThread ocsp_snowflake.py:683 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'California'), ('locality_name', 'San Mateo'), ('organization_name', 'Snowflake Inc.'), ('common_name', '*.europe-west4.gcp.snowflakecomputing.com')])
2022-05-10 16:46:36,921 - MainThread ocsp_snowflake.py:1237 - validate_by_direct_connection() - DEBUG - using OCSP response cache
2022-05-10 16:46:36,921 - MainThread ocsp_asn1crypto.py:329 - process_ocsp_response() - DEBUG - Certificate is NOT attached in Basic OCSP Response. Using issuer's certificate
2022-05-10 16:46:36,921 - MainThread ocsp_asn1crypto.py:337 - process_ocsp_response() - DEBUG - Verifying the OCSP response is signed by the issuer.
2022-05-10 16:46:36,949 - MainThread ocsp_snowflake.py:683 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert TLS RSA SHA256 2020 CA1')])
2022-05-10 16:46:36,950 - MainThread ocsp_snowflake.py:1237 - validate_by_direct_connection() - DEBUG - using OCSP response cache
2022-05-10 16:46:36,950 - MainThread ocsp_asn1crypto.py:329 - process_ocsp_response() - DEBUG - Certificate is NOT attached in Basic OCSP Response. Using issuer's certificate
2022-05-10 16:46:36,950 - MainThread ocsp_asn1crypto.py:337 - process_ocsp_response() - DEBUG - Verifying the OCSP response is signed by the issuer.
2022-05-10 16:46:36,952 - MainThread ocsp_snowflake.py:605 - write_ocsp_response_cache_file() - DEBUG - writing OCSP response cache file to /Users/damianbarabonkov/Library/Caches/Snowflake/ocsp_response_cache.json
2022-05-10 16:46:36,952 - MainThread ocsp_snowflake.py:1715 - encode_ocsp_response_cache() - DEBUG - encoding OCSP response cache to JSON
2022-05-10 16:46:36,966 - MainThread ocsp_snowflake.py:1115 - _validate() - DEBUG - ok
2022-05-10 16:46:37,299 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /session/v1/login-request?request_id=a7f15ae3-9caa-4f3f-bc14-5ee960812e11&databaseName=QUANTCORE_THEK&schemaName=PUBLIC&warehouse=THEK&roleName=ACCOUNTADMIN&request_guid=bf7ef91d-4119-412a-ad67-f367e7195523 HTTP/1.1" 200 None
2022-05-10 16:46:37,302 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:37,303 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:37,303 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:37,303 - MainThread auth.py:373 - authenticate() - DEBUG - completed authentication
2022-05-10 16:46:37,303 - MainThread auth.py:416 - authenticate() - DEBUG - token = ******
2022-05-10 16:46:37,303 - MainThread auth.py:419 - authenticate() - DEBUG - master_token = ******
2022-05-10 16:46:37,303 - MainThread auth.py:423 - authenticate() - DEBUG - id_token = NULL
2022-05-10 16:46:37,303 - MainThread auth.py:427 - authenticate() - DEBUG - mfa_token = NULL
2022-05-10 16:46:37,304 - MainThread connection.py:576 - cursor() - DEBUG - cursor
2022-05-10 16:46:37,304 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:37,304 - MainThread cursor.py:654 - execute() - DEBUG - binding: [CREATE or REPLACE TABLE QUANTCORE_THEK.PUBLIC.TEST_TABLE (CHUNK BIGINT)] with input=[None], processed=[{}]
2022-05-10 16:46:37,304 - MainThread cursor.py:700 - execute() - INFO - query: [CREATE or REPLACE TABLE QUANTCORE_THEK.PUBLIC.TEST_TABLE (CHUNK BIGINT)]
2022-05-10 16:46:37,304 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 1
2022-05-10 16:46:37,304 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: 02345ff6-63ab-4e9c-8131-9202fd5ce979
2022-05-10 16:46:37,305 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [CREATE or REPLACE TABLE QUANTCORE_THEK.PUBLIC.TEST_TABLE (CHUNK BIGINT)]
2022-05-10 16:46:37,305 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:37,305 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:37,305 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[CREATE or REPLACE TABLE QUANTCORE_THEK.PUBLIC.TEST_TABLE (CHUNK BIGINT)], sequence_id=[1], is_file_transfer=[False]
2022-05-10 16:46:37,305 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:37,305 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:37,306 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: b99617c0-5e4e-48d4-91ee-8110a8370d9e
2022-05-10 16:46:37,306 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:37,444 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=02345ff6-63ab-4e9c-8131-9202fd5ce979&request_guid=b99617c0-5e4e-48d4-91ee-8110a8370d9e HTTP/1.1" 200 None
2022-05-10 16:46:37,447 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:37,447 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:37,447 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:37,447 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52b5-0000-7ab5001a33e6
2022-05-10 16:46:37,448 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52b5-0000-7ab5001a33e6
2022-05-10 16:46:37,448 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:37,448 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:37,448 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:37,448 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: json
2022-05-10 16:46:37,449 - MainThread result_batch.py:446 - _parse() - DEBUG - parsing for result batch id: 1
2022-05-10 16:46:37,449 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:37,449 - MainThread cursor.py:654 - execute() - DEBUG - binding: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (0)] with input=[None], processed=[{}]
2022-05-10 16:46:37,449 - MainThread cursor.py:700 - execute() - INFO - query: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (0)]
2022-05-10 16:46:37,449 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 2
2022-05-10 16:46:37,449 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: bb055317-28c2-41f5-a073-44e5be9b6ca0
2022-05-10 16:46:37,450 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (0)]
2022-05-10 16:46:37,450 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:37,450 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:37,450 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (0)], sequence_id=[2], is_file_transfer=[False]
2022-05-10 16:46:37,450 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:37,450 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:37,451 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: 384479c3-33f9-4fe9-8141-62e19747f011
2022-05-10 16:46:37,451 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:38,322 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=bb055317-28c2-41f5-a073-44e5be9b6ca0&request_guid=384479c3-33f9-4fe9-8141-62e19747f011 HTTP/1.1" 200 None
2022-05-10 16:46:38,323 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:38,323 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:38,324 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:38,324 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52d0-0000-7ab5001a150e
2022-05-10 16:46:38,324 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52d0-0000-7ab5001a150e
2022-05-10 16:46:38,324 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:38,324 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:38,324 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:38,324 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: json
2022-05-10 16:46:38,325 - MainThread result_batch.py:446 - _parse() - DEBUG - parsing for result batch id: 1
2022-05-10 16:46:38,325 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:38,325 - MainThread cursor.py:654 - execute() - DEBUG - binding: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (1)] with input=[None], processed=[{}]
2022-05-10 16:46:38,325 - MainThread cursor.py:700 - execute() - INFO - query: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (1)]
2022-05-10 16:46:38,325 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 3
2022-05-10 16:46:38,325 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: 441b8db8-9c10-4804-b406-2e007eeedc5f
2022-05-10 16:46:38,325 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (1)]
2022-05-10 16:46:38,325 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:38,325 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:38,326 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (1)], sequence_id=[3], is_file_transfer=[False]
2022-05-10 16:46:38,326 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:38,326 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:38,326 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: 114a6b7d-551a-4d4a-9bd8-91723275b1cc
2022-05-10 16:46:38,326 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:38,833 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=441b8db8-9c10-4804-b406-2e007eeedc5f&request_guid=114a6b7d-551a-4d4a-9bd8-91723275b1cc HTTP/1.1" 200 None
2022-05-10 16:46:38,836 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:38,836 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:38,836 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:38,836 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52d0-0000-7ab5001a1512
2022-05-10 16:46:38,837 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52d0-0000-7ab5001a1512
2022-05-10 16:46:38,837 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:38,837 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:38,837 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:38,837 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: json
2022-05-10 16:46:38,837 - MainThread result_batch.py:446 - _parse() - DEBUG - parsing for result batch id: 1
2022-05-10 16:46:38,838 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:38,838 - MainThread cursor.py:654 - execute() - DEBUG - binding: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (2)] with input=[None], processed=[{}]
2022-05-10 16:46:38,838 - MainThread cursor.py:700 - execute() - INFO - query: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (2)]
2022-05-10 16:46:38,838 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 4
2022-05-10 16:46:38,838 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: b1b47f54-587c-491a-921f-0c4c615b2831
2022-05-10 16:46:38,838 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (2)]
2022-05-10 16:46:38,838 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:38,838 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:38,838 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (2)], sequence_id=[4], is_file_transfer=[False]
2022-05-10 16:46:38,839 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:38,839 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:38,839 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: a9e7a0fc-d26b-44f9-b7a4-f2f07edc9670
2022-05-10 16:46:38,839 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:39,347 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=b1b47f54-587c-491a-921f-0c4c615b2831&request_guid=a9e7a0fc-d26b-44f9-b7a4-f2f07edc9670 HTTP/1.1" 200 None
2022-05-10 16:46:39,350 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:39,350 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:39,350 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:39,350 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52d0-0000-7ab5001a1516
2022-05-10 16:46:39,351 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52d0-0000-7ab5001a1516
2022-05-10 16:46:39,351 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:39,351 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:39,351 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:39,351 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: json
2022-05-10 16:46:39,351 - MainThread result_batch.py:446 - _parse() - DEBUG - parsing for result batch id: 1
2022-05-10 16:46:39,351 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:39,352 - MainThread cursor.py:654 - execute() - DEBUG - binding: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (3)] with input=[None], processed=[{}]
2022-05-10 16:46:39,352 - MainThread cursor.py:700 - execute() - INFO - query: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (3)]
2022-05-10 16:46:39,352 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 5
2022-05-10 16:46:39,352 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: 7998387c-2894-458f-a668-861b9ee62b88
2022-05-10 16:46:39,352 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (3)]
2022-05-10 16:46:39,352 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:39,352 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:39,352 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (3)], sequence_id=[5], is_file_transfer=[False]
2022-05-10 16:46:39,353 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:39,353 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:39,353 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: 59873ac3-3731-40ea-b580-5a79ab75ee42
2022-05-10 16:46:39,353 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:40,135 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=7998387c-2894-458f-a668-861b9ee62b88&request_guid=59873ac3-3731-40ea-b580-5a79ab75ee42 HTTP/1.1" 200 None
2022-05-10 16:46:40,137 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:40,138 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:40,138 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:40,138 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52b5-0000-7ab5001a33ea
2022-05-10 16:46:40,139 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52b5-0000-7ab5001a33ea
2022-05-10 16:46:40,139 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:40,139 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:40,139 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:40,139 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: json
2022-05-10 16:46:40,140 - MainThread result_batch.py:446 - _parse() - DEBUG - parsing for result batch id: 1
2022-05-10 16:46:40,140 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:40,140 - MainThread cursor.py:654 - execute() - DEBUG - binding: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (4)] with input=[None], processed=[{}]
2022-05-10 16:46:40,140 - MainThread cursor.py:700 - execute() - INFO - query: [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (4)]
2022-05-10 16:46:40,140 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 6
2022-05-10 16:46:40,140 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: f2779911-f9f7-499f-9453-0d9eeb708bca
2022-05-10 16:46:40,140 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (4)]
2022-05-10 16:46:40,141 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:40,141 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:40,141 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[INSERT INTO QUANTCORE_THEK.PUBLIC.TEST_TABLE VALUES (4)], sequence_id=[6], is_file_transfer=[False]
2022-05-10 16:46:40,141 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:40,141 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:40,141 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: dbb4a73a-5efe-4788-9640-896e0c35e6bf
2022-05-10 16:46:40,141 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:40,593 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=f2779911-f9f7-499f-9453-0d9eeb708bca&request_guid=dbb4a73a-5efe-4788-9640-896e0c35e6bf HTTP/1.1" 200 None
2022-05-10 16:46:40,595 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:40,595 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:40,596 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:40,596 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52d0-0000-7ab5001a151a
2022-05-10 16:46:40,596 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52d0-0000-7ab5001a151a
2022-05-10 16:46:40,596 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:40,597 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:40,597 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:40,597 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: json
2022-05-10 16:46:40,597 - MainThread result_batch.py:446 - _parse() - DEBUG - parsing for result batch id: 1
2022-05-10 16:46:40,597 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:40,597 - MainThread cursor.py:654 - execute() - DEBUG - binding: [SELECT CHUNK FROM PUBLIC.TEST_TABLE] with input=[None], processed=[{}]
2022-05-10 16:46:40,598 - MainThread cursor.py:700 - execute() - INFO - query: [SELECT CHUNK FROM PUBLIC.TEST_TABLE]
2022-05-10 16:46:40,598 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 7
2022-05-10 16:46:40,598 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: ae7acc60-288b-4d98-8005-82c6ca3e237a
2022-05-10 16:46:40,598 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [SELECT CHUNK FROM PUBLIC.TEST_TABLE]
2022-05-10 16:46:40,598 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:40,598 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:40,598 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[SELECT CHUNK FROM PUBLIC.TEST_TABLE], sequence_id=[7], is_file_transfer=[False]
2022-05-10 16:46:40,599 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:40,599 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:40,599 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: c7cedcc8-db96-4b18-b941-ae250d9ae63e
2022-05-10 16:46:40,599 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:40,803 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=ae7acc60-288b-4d98-8005-82c6ca3e237a&request_guid=c7cedcc8-db96-4b18-b941-ae250d9ae63e HTTP/1.1" 200 None
2022-05-10 16:46:40,805 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:40,806 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:40,806 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:40,806 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52b5-0000-7ab5001a33ee
2022-05-10 16:46:40,807 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52b5-0000-7ab5001a33ee
2022-05-10 16:46:40,807 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:40,807 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:40,807 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:40,807 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: arrow
2022-05-10 16:46:40,808 - MainThread arrow_iterator.cpython-310-darwin.so:0 - __cinit__() - DEBUG - Batches read: 1
2022-05-10 16:46:40,809 - MainThread CArrowIterator.cpp:16 - CArrowIterator() - DEBUG - Arrow BatchSize: 1
2022-05-10 16:46:40,809 - MainThread CArrowChunkIterator.cpp:50 - CArrowChunkIterator() - DEBUG - Arrow chunk info: batchCount 1, columnCount 1, use_numpy: 0
2022-05-10 16:46:40,809 - MainThread result_set.py:58 - result_set_iterator() - DEBUG - beginning to schedule result batch downloads
2022-05-10 16:46:40,809 - MainThread CArrowChunkIterator.cpp:74 - next() - DEBUG - Current batch index: 0, rows in current batch: 5
   (CHUNK, 0, None, None, 38, 0, True)
0                                    0
1                                    1
2                                    2
3                                    3
4                                    4
(CHUNK, 0, None, None, 38, 0, True)    int64
dtype: object
2022-05-10 16:46:40,823 - MainThread cursor.py:616 - execute() - DEBUG - executing SQL/command
2022-05-10 16:46:40,823 - MainThread cursor.py:654 - execute() - DEBUG - binding: [SELECT CHUNK FROM PUBLIC.TEST_TABLE] with input=[None], processed=[{}]
2022-05-10 16:46:40,823 - MainThread cursor.py:700 - execute() - INFO - query: [SELECT CHUNK FROM PUBLIC.TEST_TABLE]
2022-05-10 16:46:40,823 - MainThread connection.py:1252 - _next_sequence_counter() - DEBUG - sequence counter: 8
2022-05-10 16:46:40,823 - MainThread cursor.py:447 - _execute_helper() - DEBUG - Request id: a092e1c4-e11c-405a-9a29-46d419c10c99
2022-05-10 16:46:40,823 - MainThread cursor.py:450 - _execute_helper() - DEBUG - running query [SELECT CHUNK FROM PUBLIC.TEST_TABLE]
2022-05-10 16:46:40,823 - MainThread cursor.py:459 - _execute_helper() - DEBUG - is_file_transfer: False
2022-05-10 16:46:40,823 - MainThread connection.py:923 - cmd_query() - DEBUG - _cmd_query
2022-05-10 16:46:40,823 - MainThread connection.py:946 - cmd_query() - DEBUG - sql=[SELECT CHUNK FROM PUBLIC.TEST_TABLE], sequence_id=[8], is_file_transfer=[False]
2022-05-10 16:46:40,824 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:40,824 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-05-10 16:46:40,824 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: afb0c66e-0178-4a5c-a385-983dc0ab5a4e
2022-05-10 16:46:40,824 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:40,893 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=a092e1c4-e11c-405a-9a29-46d419c10c99&request_guid=afb0c66e-0178-4a5c-a385-983dc0ab5a4e HTTP/1.1" 200 None
2022-05-10 16:46:40,894 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:40,894 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:40,894 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:40,894 - MainThread network.py:738 - _post_request() - DEBUG - Query id: 01a42cb6-0000-52b5-0000-7ab5001a33f2
2022-05-10 16:46:40,894 - MainThread cursor.py:722 - execute() - DEBUG - sfqid: 01a42cb6-0000-52b5-0000-7ab5001a33f2
2022-05-10 16:46:40,894 - MainThread cursor.py:724 - execute() - INFO - query execution done
2022-05-10 16:46:40,894 - MainThread cursor.py:726 - execute() - DEBUG - SUCCESS
2022-05-10 16:46:40,895 - MainThread cursor.py:729 - execute() - DEBUG - PUT OR GET: None
2022-05-10 16:46:40,895 - MainThread cursor.py:826 - _init_result_and_meta() - DEBUG - Query result format: arrow
2022-05-10 16:46:40,895 - MainThread arrow_iterator.cpython-310-darwin.so:0 - __cinit__() - DEBUG - Batches read: 1
2022-05-10 16:46:40,895 - MainThread CArrowIterator.cpp:16 - CArrowIterator() - DEBUG - Arrow BatchSize: 1
   CHUNK
0      0
1      1
2      2
3      3
4      4
CHUNK    int8
dtype: object
2022-05-10 16:46:40,908 - MainThread connection.py:509 - close() - INFO - closed
2022-05-10 16:46:40,908 - MainThread telemetry.py:151 - close() - DEBUG - Closing telemetry client.
2022-05-10 16:46:40,909 - MainThread telemetry.py:116 - send_batch() - DEBUG - Sending 1 logs to telemetry. Data is {'logs': [{'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52b5-0000-7ab5001a33e6', 'value': 6}, 'timestamp': '1652193997448'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52d0-0000-7ab5001a150e', 'value': 30}, 'timestamp': '1652193998324'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52d0-0000-7ab5001a1512', 'value': 37}, 'timestamp': '1652193998837'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52d0-0000-7ab5001a1516', 'value': 42}, 'timestamp': '1652193999351'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52b5-0000-7ab5001a33ea', 'value': 7}, 'timestamp': '1652194000139'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52d0-0000-7ab5001a151a', 'value': 5}, 'timestamp': '1652194000596'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52b5-0000-7ab5001a33ee', 'value': 5}, 'timestamp': '1652194000807'}, {'message': {'type': 'client_time_consume_last_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52b5-0000-7ab5001a33ee', 'value': 2}, 'timestamp': '1652194000809'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52b5-0000-7ab5001a33f2', 'value': 1}, 'timestamp': '1652194000894'}, {'message': {'type': 'client_get_partitions_used', 'source': 'PythonConnector', 'query_id': '01a42cb6-0000-52b5-0000-7ab5001a33f2', 'value': 1}, 'timestamp': '1652194000895'}]}.
2022-05-10 16:46:40,910 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:40,910 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: 5, retry cnt: 1
2022-05-10 16:46:40,910 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: 76729319-e312-40a9-81c9-0567a3cfe4cb
2022-05-10 16:46:40,910 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:40,953 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /telemetry/send?request_guid=76729319-e312-40a9-81c9-0567a3cfe4cb HTTP/1.1" 200 86
2022-05-10 16:46:40,954 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:40,954 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:40,954 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:40,954 - MainThread telemetry.py:140 - send_batch() - DEBUG - Successfully uploading metrics to telemetry.
2022-05-10 16:46:40,954 - MainThread connection.py:512 - close() - INFO - No async queries seem to be running, deleting session
2022-05-10 16:46:40,954 - MainThread network.py:1120 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 1/1 active sessions
2022-05-10 16:46:40,954 - MainThread network.py:826 - _request_exec_wrapper() - DEBUG - remaining request timeout: 5, retry cnt: 1
2022-05-10 16:46:40,954 - MainThread network.py:807 - add_request_guid() - DEBUG - Request guid: f1b47abd-7121-4b96-9727-a44baa51569b
2022-05-10 16:46:40,954 - MainThread network.py:976 - _request_exec() - DEBUG - socket timeout: 60
2022-05-10 16:46:41,022 - MainThread connectionpool.py:452 - _make_request() - DEBUG - https://ag44040.europe-west4.gcp.snowflakecomputing.com:443 "POST /session?delete=true&request_guid=f1b47abd-7121-4b96-9727-a44baa51569b HTTP/1.1" 200 76
2022-05-10 16:46:41,022 - MainThread network.py:1005 - _request_exec() - DEBUG - SUCCESS
2022-05-10 16:46:41,023 - MainThread network.py:1125 - _use_requests_session() - DEBUG - Session status for SessionPool 'ag44040.europe-west4.gcp.snowflakecomputing.com', SessionPool 0/1 active sessions
2022-05-10 16:46:41,023 - MainThread network.py:714 - _post_request() - DEBUG - ret[code] = None, after post request
2022-05-10 16:46:41,026 - MainThread connection.py:523 - close() - DEBUG - Session is closed
2022-05-10 16:46:41,076 - MainThread connection.py:500 - close() - DEBUG - Rest object has been destroyed, cannot close session

DamianBarabonkovQC avatar May 10 '22 14:05 DamianBarabonkovQC

Hi @DamianBarabonkovQC This isn't a bug, as our docs say: INT , INTEGER , BIGINT , SMALLINT , TINYINT , BYTEINT are all just synonymous with NUMBER. So this type has no special meaning. Otherwise Snowflake is smart enough to keep some metadata like maximum and minimum numbers of numerical columns. So when you fetch data the Python connector will get INT8 from our back-end based on the extra metadata it has access to. That's not to say that you can't lose the metadata in more complicatedqueries:

def fetch_pandas_also_returns_int64(cur: snowflake.connector.cursor.SnowflakeCursor, sql: str) -> pd.DataFrame:
    cur.execute(sql)
    batches = cur.execute("select 1::varchar::bigint from table(result_scan())").get_result_batches()
    return batches[0].to_pandas()

If your code requires INT64 then could I suggest force upcasting your dataframe's columns?

df = df.astype({"CHUNK": "int64"})
print(df)
print(df.dtypes)

sfc-gh-mkeller avatar May 12 '22 04:05 sfc-gh-mkeller

@sfc-gh-mkeller Sorry if this is not the right place to ask this question. If you run a describe with the Python connector does it provide this metadata as well? If so what part of the ResultMetadata does it correspond to? https://docs.snowflake.com/en/user-guide/python-connector-api.html#object-resultmetadata

njriasan avatar May 13 '22 14:05 njriasan

@njriasan in this case the metadata will be the same for all 3 ways of execution:

[ResultMetadata(name='CHUNK', type_code=0, display_size=None, internal_size=None, precision=38, scale=0, is_nullable=True)]

This is because the the schema of the results is a column named CHUNK with type NUMBER(38,0) (this is what cur.description tells you). It just happens that sometime NUMBER(38,0) can fit into an INT8, INT64, all the way up to a DECIMAL128.

Hope this is helpful!

edit: let me be a little more explicit. NUMBER(38,0) mean that it's a number with up to 38 significant figures and 0 decimal figures. So 1 and 9,223,372,036,854,775,808 both fit into the definition of NUMBER(38,0). However; 1 will fit pyarrow.int8 while the other one won't. So if the results only have small enough numbers to each fit into 8 bits only then to save bandwidth (and accelerate execution) Snowflake will not pad the values any more than it needs to.

sfc-gh-mkeller avatar May 17 '22 23:05 sfc-gh-mkeller

@sfc-gh-mkeller Thanks for the response. I was hoping that based on your comment "will get INT8 from our back-end based on the extra metadata it has access to" that it might be possible to access this metadata as part of describe (at least for simple queries that just select columns). I'm guessing the metadata you are referring to here may be the result of statistically information like the max and min after performing certain computes (and why it can't be included in describe).

For context I work on an execution engine, Bodo, and would like to explicitly produced compiled code to use this lower bit width from describe whenever possible.

njriasan avatar May 18 '22 00:05 njriasan

@njriasan I don't think that our back-end exposes this metadata to clients. Feel free to open a feature request for this, but it probably won't happen in the short term to be honest with you.

edit: It was brought to my attention that you can get the how many bytes a column fits into with the TYPEOF command

sfc-gh-mkeller avatar May 31 '22 18:05 sfc-gh-mkeller

To clean up and re-prioritize bugs and feature requests we are closing all issues older than 6 months as of March 1, 2023. If there are any issues or feature requests that you would like us to address, please re-create them. For urgent issues, opening a support case with this link Snowflake Community is the fastest way to get a response

github-actions[bot] avatar Mar 10 '23 02:03 github-actions[bot]