snowflake-connector-python SNOW-647590: Using `execute_async` to perform a `PUT` command raises KeyError

SNOW-647590: Using `execute_async` to perform a `PUT` command raises KeyError

Open kandji-ross opened this issue 3 years ago • 2 comments

Please answer these questions before submitting your issue. Thanks!

What version of Python are you using?

python --version --version
Python 3.9.7 (default, Nov 15 2021, 14:50:37)
[Clang 12.0.5 (clang-1205.0.22.11)]

What operating system and processor architecture are you using?

python -c 'import platform; print(platform.platform())'
macOS-12.4-x86_64-i386-64bit

What are the component versions in the environment (pip freeze)?

python -m pip freeze
asn1crypto==1.5.1
certifi==2022.6.15
cffi==1.15.1
charset-normalizer==2.1.0
cryptography==36.0.2
idna==3.3
oscrypto==1.3.0
pycparser==2.21
pycryptodomex==3.15.0
PyJWT==2.4.0
pyOpenSSL==22.0.0
pytz==2022.2.1
requests==2.28.1
snowflake-connector-python==2.7.11
typing_extensions==4.3.0
urllib3==1.26.11

What did you do?

Attempted to run a PUT command via execute_async.

What did you expect to see?

Expected that the command would work without error.

A simple demo script like:

conn = snowflake.connector.connect(
    user=SNOW_USER,
    password=SNOW_PASSWORD,
    account=SNOW_ACCOUNT,
    database=SNOW_DB,
    warehouse=SNOW_WAREHOUSE,
    role=SNOW_ROLE,
)

cur = conn.cursor()

filepath = '...'
schema = '...'
table = '...'

cur.execute_async(f"PUT {filepath} @{schema}.%{table}")

produces a traceback like:

Traceback (most recent call last):
  File "...", line 35, in <module>
    cur.execute_async(f"PUT {filepath} @{schema}.%{table}")
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 814, in execute_async
    return self.execute(*args, **kwargs)
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 765, in execute
    sf_file_transfer_agent.execute()
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 358, in execute
    self._parse_command()
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 858, in _parse_command
    self._command_type = response["command"]
KeyError: 'command'

If I wrap the execute_async call in a try/except block, it does work as expected and I can retrieve the queryid from the cursor and proceed as normal, but that feels rather awkward:

try:
    cur.execute_async(f"PUT {filepath} @{schema}.%{table}")
except KeyError:
  pass
query_id = cur.sfqid
while conn.is_still_running(conn.get_query_status(query_id)):
  time.sleep(1)

Can you set logging to DEBUG and collect the logs?

2022-08-18 09:44:57,290 - MainThread connection.py:270 - __init__() - INFO - Snowflake Connector for Python Version: 2.7.11, Python Version: 3.9.7, Platform: macOS-12.4-x86_64-i386-64bit
2022-08-18 09:44:57,291 - MainThread connection.py:512 - connect() - DEBUG - connect
2022-08-18 09:44:57,291 - MainThread connection.py:802 - __config() - DEBUG - __config
2022-08-18 09:44:57,291 - MainThread connection.py:926 - __config() - INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
2022-08-18 09:44:57,291 - MainThread connection.py:944 - __config() - INFO - Setting use_openssl_only mode to False
2022-08-18 09:44:57,291 - MainThread converter.py:145 - __init__() - DEBUG - use_numpy: False
2022-08-18 09:44:57,291 - MainThread connection.py:705 - __open_connection() - DEBUG - REST API object was created: [account].snowflakecomputing.com:443
2022-08-18 09:44:57,291 - MainThread auth.py:170 - authenticate() - DEBUG - authenticate
2022-08-18 09:44:57,291 - MainThread auth.py:200 - authenticate() - DEBUG - assertion content: *********
2022-08-18 09:44:57,291 - MainThread auth.py:203 - authenticate() - DEBUG - account=[account], user=[user], database=[database], schema=None, warehouse=[warehouse], role=[role], request_id=1666ca14-4388-44c9-a651-54ef5175947c
2022-08-18 09:44:57,291 - MainThread auth.py:236 - authenticate() - DEBUG - body['data']: {'CLIENT_APP_ID': 'PythonConnector', 'CLIENT_APP_VERSION': '2.7.11', 'SVN_REVISION': None, 'ACCOUNT_NAME': '[account]', 'LOGIN_NAME': '[username]', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'PythonConnector', 'OS': 'Darwin', 'OS_VERSION': 'macOS-12.4-x86_64-i386-64bit', 'PYTHON_VERSION': '3.9.7', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'Clang 12.0.5 (clang-1205.0.22.11)', 'OCSP_MODE': 'FAIL_OPEN', 'TRACING': 10, 'LOGIN_TIMEOUT': 120, 'NETWORK_TIMEOUT': None}, 'SESSION_PARAMETERS': {'CLIENT_PREFETCH_THREADS': 4}}
2022-08-18 09:44:57,291 - MainThread auth.py:254 - authenticate() - DEBUG - Timeout set to 120
2022-08-18 09:44:57,292 - MainThread retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2022-08-18 09:44:57,292 - MainThread retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2022-08-18 09:44:57,292 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool '[account].snowflakecomputing.com', SessionPool 1/1 active sessions
2022-08-18 09:44:57,292 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: 120, retry cnt: 1
2022-08-18 09:44:57,292 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: cc340cc8-0a49-4e42-a8b6-8e425a9aaef8
2022-08-18 09:44:57,292 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2022-08-18 09:44:57,406 - MainThread connectionpool.py:1003 - _new_conn() - DEBUG - Starting new HTTPS connection (1): [account].snowflakecomputing.com:443
2022-08-18 09:44:57,839 - MainThread ocsp_snowflake.py:490 - reset_cache_dir() - DEBUG - cache directory: .../Library/Caches/Snowflake
2022-08-18 09:44:57,939 - MainThread ssl_wrap_socket.py:80 - ssl_wrap_socket_with_ocsp() - DEBUG - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
2022-08-18 09:44:57,940 - MainThread ocsp_snowflake.py:532 - reset_ocsp_response_cache_uri() - DEBUG - ocsp_response_cache_uri: file://.../Library/Caches/Snowflake/ocsp_response_cache.json
2022-08-18 09:44:57,940 - MainThread ocsp_snowflake.py:533 - reset_ocsp_response_cache_uri() - DEBUG - OCSP_VALIDATION_CACHE size: 0
2022-08-18 09:44:57,940 - MainThread ocsp_snowflake.py:315 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP response cache server is enabled: http://ocsp.snowflakecomputing.com/ocsp_response_cache.json
2022-08-18 09:44:57,940 - MainThread ocsp_snowflake.py:351 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP dynamic cache server RETRY URL: None
2022-08-18 09:44:57,943 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2022-08-18 09:44:57,944 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
2022-08-18 09:44:57,946 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2022-08-18 09:44:57,947 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2022-08-18 09:44:57,950 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2022-10-15 00:00:00+00:00
2022-08-18 09:44:57,968 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:57,969 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,970 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,971 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:57,973 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:57,974 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,975 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:57,978 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,979 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,982 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,985 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:57,996 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,998 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:57,999 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,000 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:58,002 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:58,004 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,007 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,009 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,010 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,013 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,014 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,016 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:58,017 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:58,019 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 15:48:31+00:00
2022-08-18 09:44:58,020 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:58,022 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:58,024 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-06-22 07:00:00+00:00
2022-08-18 09:44:58,025 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-07-15 16:47:06+00:00
2022-08-18 09:44:58,026 - MainThread ocsp_snowflake.py:568 - read_ocsp_response_cache_file() - DEBUG - Read OCSP response cache file: .../Library/Caches/Snowflake/ocsp_response_cache.json, count=194
2022-08-18 09:44:58,026 - MainThread ocsp_snowflake.py:1101 - validate() - DEBUG - validating certificate: [account].snowflakecomputing.com
2022-08-18 09:44:58,026 - MainThread ocsp_asn1crypto.py:439 - extract_certificate_chain() - DEBUG - # of certificates: 4
2022-08-18 09:44:58,026 - MainThread ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('common_name', '*.us-west-2.snowflakecomputing.com')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('organizational_unit_name', 'Server CA 1B'), ('common_name', 'Amazon')])
2022-08-18 09:44:58,027 - MainThread ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('organizational_unit_name', 'Server CA 1B'), ('common_name', 'Amazon')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')])
2022-08-18 09:44:58,027 - MainThread ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')]), issuer: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')])
2022-08-18 09:44:58,028 - MainThread ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Starfield Technologies, Inc.'), ('organizational_unit_name', 'Starfield Class 2 Certification Authority')])
2022-08-18 09:44:58,030 - MainThread ocsp_asn1crypto.py:123 - read_cert_bundle() - DEBUG - reading certificate bundle: .../.venv/lib/python3.9/site-packages/certifi/cacert.pem
2022-08-18 09:44:58,055 - MainThread ocsp_asn1crypto.py:465 - create_pair_issuer_subject() - DEBUG - not found issuer_der: OrderedDict([('country_name', 'US'), ('organization_name', 'Starfield Technologies, Inc.'), ('organizational_unit_name', 'Starfield Class 2 Certification Authority')])
2022-08-18 09:44:58,056 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('common_name', '*.us-west-2.snowflakecomputing.com')])
2022-08-18 09:44:58,058 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2022-08-18 09:44:58,058 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('organizational_unit_name', 'Server CA 1B'), ('common_name', 'Amazon')])
2022-08-18 09:44:58,060 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
2022-08-18 09:44:58,060 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')])
2022-08-18 09:44:58,063 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-06-22 07:00:00+00:00
2022-08-18 09:44:58,063 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')])
2022-08-18 09:44:58,064 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('common_name', '*.us-west-2.snowflakecomputing.com')])
2022-08-18 09:44:58,065 - MainThread ocsp_snowflake.py:1285 - validate_by_direct_connection() - DEBUG - using OCSP response cache
2022-08-18 09:44:58,066 - MainThread ocsp_asn1crypto.py:333 - process_ocsp_response() - DEBUG - Certificate is NOT attached in Basic OCSP Response. Using issuer's certificate
2022-08-18 09:44:58,066 - MainThread ocsp_asn1crypto.py:341 - process_ocsp_response() - DEBUG - Verifying the OCSP response is signed by the issuer.
2022-08-18 09:44:58,107 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2022-08-18 09:44:58,107 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('organizational_unit_name', 'Server CA 1B'), ('common_name', 'Amazon')])
2022-08-18 09:44:58,109 - MainThread ocsp_snowflake.py:1285 - validate_by_direct_connection() - DEBUG - using OCSP response cache
2022-08-18 09:44:58,110 - MainThread ocsp_asn1crypto.py:298 - process_ocsp_response() - DEBUG - Certificate is attached in Basic OCSP Response
2022-08-18 09:44:58,110 - MainThread ocsp_asn1crypto.py:300 - process_ocsp_response() - DEBUG - Verifying the attached certificate is signed by the issuer
2022-08-18 09:44:58,110 - MainThread ocsp_asn1crypto.py:303 - process_ocsp_response() - DEBUG - Valid Not After: 2025-09-15 19:41:06+00:00
2022-08-18 09:44:58,112 - MainThread ocsp_asn1crypto.py:341 - process_ocsp_response() - DEBUG - Verifying the OCSP response is signed by the issuer.
2022-08-18 09:44:58,116 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
2022-08-18 09:44:58,117 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')])
2022-08-18 09:44:58,118 - MainThread ocsp_snowflake.py:1285 - validate_by_direct_connection() - DEBUG - using OCSP response cache
2022-08-18 09:44:58,119 - MainThread ocsp_asn1crypto.py:298 - process_ocsp_response() - DEBUG - Certificate is attached in Basic OCSP Response
2022-08-18 09:44:58,119 - MainThread ocsp_asn1crypto.py:300 - process_ocsp_response() - DEBUG - Verifying the attached certificate is signed by the issuer
2022-08-18 09:44:58,119 - MainThread ocsp_asn1crypto.py:303 - process_ocsp_response() - DEBUG - Valid Not After: 2025-09-15 19:45:34+00:00
2022-08-18 09:44:58,122 - MainThread ocsp_asn1crypto.py:341 - process_ocsp_response() - DEBUG - Verifying the OCSP response is signed by the issuer.
2022-08-18 09:44:58,129 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-06-22 07:00:00+00:00
2022-08-18 09:44:58,129 - MainThread ocsp_snowflake.py:708 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')])
2022-08-18 09:44:58,130 - MainThread ocsp_snowflake.py:1285 - validate_by_direct_connection() - DEBUG - using OCSP response cache
2022-08-18 09:44:58,131 - MainThread ocsp_asn1crypto.py:298 - process_ocsp_response() - DEBUG - Certificate is attached in Basic OCSP Response
2022-08-18 09:44:58,131 - MainThread ocsp_asn1crypto.py:300 - process_ocsp_response() - DEBUG - Verifying the attached certificate is signed by the issuer
2022-08-18 09:44:58,132 - MainThread ocsp_asn1crypto.py:303 - process_ocsp_response() - DEBUG - Valid Not After: 2023-06-22 07:00:00+00:00
2022-08-18 09:44:58,135 - MainThread ocsp_asn1crypto.py:341 - process_ocsp_response() - DEBUG - Verifying the OCSP response is signed by the issuer.
2022-08-18 09:44:58,137 - MainThread ocsp_snowflake.py:630 - write_ocsp_response_cache_file() - DEBUG - writing OCSP response cache file to .../Library/Caches/Snowflake/ocsp_response_cache.json
2022-08-18 09:44:58,137 - MainThread ocsp_snowflake.py:1777 - encode_ocsp_response_cache() - DEBUG - encoding OCSP response cache to JSON
2022-08-18 09:44:58,165 - MainThread ocsp_snowflake.py:1158 - _validate() - DEBUG - ok
2022-08-18 09:44:58,517 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://[account].snowflakecomputing.com:443 "POST /session/v1/login-request?request_id=1666ca14-4388-44c9-a651-54ef5175947c&databaseName=[database]&warehouse=[warehouse]&roleName=[role]&request_guid=cc340cc8-0a49-4e42-a8b6-8e425a9aaef8 HTTP/1.1" 200 1485
2022-08-18 09:44:58,517 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2022-08-18 09:44:58,517 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool '[account].snowflakecomputing.com', SessionPool 0/1 active sessions
2022-08-18 09:44:58,517 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2022-08-18 09:44:58,517 - MainThread auth.py:373 - authenticate() - DEBUG - completed authentication
2022-08-18 09:44:58,518 - MainThread auth.py:416 - authenticate() - DEBUG - token = ******
2022-08-18 09:44:58,518 - MainThread auth.py:419 - authenticate() - DEBUG - master_token = ******
2022-08-18 09:44:58,518 - MainThread auth.py:423 - authenticate() - DEBUG - id_token = NULL
2022-08-18 09:44:58,518 - MainThread auth.py:427 - authenticate() - DEBUG - mfa_token = NULL
2022-08-18 09:44:58,518 - MainThread connection.py:624 - cursor() - DEBUG - cursor
2022-08-18 09:44:58,518 - MainThread cursor.py:630 - execute() - DEBUG - executing SQL/command
2022-08-18 09:44:58,518 - MainThread cursor.py:669 - execute() - DEBUG - binding: [PUT file://[filename.json] @[schema].%[table]] with input=[None], processed=[{}]
2022-08-18 09:44:58,519 - MainThread cursor.py:715 - execute() - INFO - query: [PUT file://[filename.json] @[schema].%[table]]
2022-08-18 09:44:58,519 - MainThread connection.py:1302 - _next_sequence_counter() - DEBUG - sequence counter: 1
2022-08-18 09:44:58,519 - MainThread cursor.py:460 - _execute_helper() - DEBUG - Request id: 5b549243-a59a-4622-b306-de2353938157
2022-08-18 09:44:58,519 - MainThread cursor.py:462 - _execute_helper() - DEBUG - running query [PUT file://[filename.json] @[schema].%[table]]
2022-08-18 09:44:58,519 - MainThread cursor.py:471 - _execute_helper() - DEBUG - is_file_transfer: True
2022-08-18 09:44:58,519 - MainThread connection.py:972 - cmd_query() - DEBUG - _cmd_query
2022-08-18 09:44:58,519 - MainThread connection.py:995 - cmd_query() - DEBUG - sql=[PUT file://[filename.json] @[schema].%[table]], sequence_id=[1], is_file_transfer=[True]
2022-08-18 09:44:58,519 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool '[account].snowflakecomputing.com', SessionPool 1/1 active sessions
2022-08-18 09:44:58,519 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2022-08-18 09:44:58,519 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: 5aa7853f-73d2-4efb-8d9f-4bed9e72d01e
2022-08-18 09:44:58,519 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2022-08-18 09:44:58,647 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://[account].snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=5b549243-a59a-4622-b306-de2353938157&request_guid=5aa7853f-73d2-4efb-8d9f-4bed9e72d01e HTTP/1.1" 200 363
2022-08-18 09:44:58,647 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2022-08-18 09:44:58,648 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool '[account].snowflakecomputing.com', SessionPool 0/1 active sessions
2022-08-18 09:44:58,648 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = 333334, after post request
2022-08-18 09:44:58,648 - MainThread network.py:739 - _post_request() - DEBUG - Query id: 01a65ef8-0401-bb7d-005e-51830015a1a6
2022-08-18 09:44:58,648 - MainThread cursor.py:737 - execute() - DEBUG - sfqid: 01a65ef8-0401-bb7d-005e-51830015a1a6
2022-08-18 09:44:58,648 - MainThread cursor.py:739 - execute() - INFO - query execution done
2022-08-18 09:44:58,648 - MainThread cursor.py:741 - execute() - DEBUG - SUCCESS
2022-08-18 09:44:58,648 - MainThread cursor.py:744 - execute() - DEBUG - PUT OR GET: <re.Match object; span=(0, 4), match='PUT '>
Traceback (most recent call last):
  File ".../Untitled", line 38, in <module>
    cur.execute_async(f"PUT {filepath} @{schema}.%{table}")
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 814, in execute_async
    return self.execute(*args, **kwargs)
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 765, in execute
    sf_file_transfer_agent.execute()
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 358, in execute
    self._parse_command()
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/file_transfer_agent.py", line 858, in _parse_command
    self._command_type = response["command"]
KeyError: 'command'
2022-08-18 09:44:58,673 - MainThread connection.py:557 - close() - INFO - closed
2022-08-18 09:44:58,673 - MainThread telemetry.py:151 - close() - DEBUG - Closing telemetry client.
2022-08-18 09:44:58,673 - MainThread connection.py:560 - close() - INFO - No async queries seem to be running, deleting session
2022-08-18 09:44:58,673 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool '[account].snowflakecomputing.com', SessionPool 1/1 active sessions
2022-08-18 09:44:58,673 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: 5, retry cnt: 1
2022-08-18 09:44:58,674 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: b448c751-b888-454b-963a-c44b5cd31db3
2022-08-18 09:44:58,674 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2022-08-18 09:44:58,679 - MainThread network.py:918 - _request_exec_wrapper() - DEBUG - Ignored error
Traceback (most recent call last):
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/network.py", line 837, in _request_exec_wrapper
    return_object = self._request_exec(
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/network.py", line 1116, in _request_exec
    raise err
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/network.py", line 1018, in _request_exec
    raw_ret = session.request(
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/vendored/requests/sessions.py", line 577, in request
    settings = self.merge_environment_settings(
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/vendored/requests/sessions.py", line 759, in merge_environment_settings
    env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/vendored/requests/utils.py", line 825, in get_environ_proxies
    if should_bypass_proxies(url, no_proxy=no_proxy):
  File ".../.venv/lib/python3.9/site-packages/snowflake/connector/vendored/requests/utils.py", line 809, in should_bypass_proxies
    bypass = proxy_bypass(parsed.hostname)
  File ".../.pyenv/versions/3.9.7/lib/python3.9/urllib/request.py", line 2647, in proxy_bypass
    return proxy_bypass_macosx_sysconf(host)
  File ".../.pyenv/versions/3.9.7/lib/python3.9/urllib/request.py", line 2624, in proxy_bypass_macosx_sysconf
    return _proxy_bypass_macosx_sysconf(host, proxy_settings)
  File ".../.pyenv/versions/3.9.7/lib/python3.9/urllib/request.py", line 2566, in _proxy_bypass_macosx_sysconf
    from fnmatch import fnmatch
ImportError: sys.meta_path is None, Python is likely shutting down
2022-08-18 09:44:58,679 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool '[account].snowflakecomputing.com', SessionPool 0/1 active sessions
2022-08-18 09:44:58,679 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = N/A, after post request
2022-08-18 09:44:58,680 - MainThread connection.py:571 - close() - DEBUG - Session is closed

Aug 18 '22 13:08 kandji-ross

I'm seeing a very similar issue. Any updates on this?

Oct 03 '22 14:10 sahil1105

PUT and GET statements are just not supported by execute_async. The problem is that we'd need a SQL parser to find them. However; we should probably improve our documentation here to include this information.

Would one of you like to open a PR for this?

Oct 17 '22 20:10 sfc-gh-mkeller

PUT and GET statements are just not supported by execute_async. The problem is that we'd need a SQL parser to find them. However; we should probably improve our documentation here to include this information.

Would one of you like to open a PR for this?

Hi @sfc-gh-mkeller, I've updated the docstring for the execute_async method, specifying that GET and PUT statements are not supported by this method. I hope this is what you intended, I've created a PR.

Jan 13 '23 05:01 ben-nour

snowflake-connector-python snowflake-connector-python copied to clipboard

SNOW-647590: Using `execute_async` to perform a `PUT` command raises KeyError

snowflake-connector-python
snowflake-connector-python copied to clipboard