snowflake-connector-python icon indicating copy to clipboard operation
snowflake-connector-python copied to clipboard

SNOW-1511473: "type[type]" is incompatible with "type[SnowflakeCursor]"

Open tekumara opened this issue 1 year ago • 5 comments

Python version

3.9

Operating system and processor architecture

macos

Installed packages

snowflake-connector-python==3.11.0

What did you do?

*_, dcur = conn.execute_string("select 1",cursor_class=snowflake.connector.cursor.DictCursor)
dcur.fetchall()

What did you expect to see?

No type error, but get this in pyright/pylance/vscode:

CleanShot 2024-06-28 at 22 11 56@2x

Can you set logging to DEBUG and collect the logs?

No response

tekumara avatar Jun 28 '24 12:06 tekumara

Hello @tekumara ,

Thanks for raising the issue, we are looking into it, will update.

Regards, Sujan

sfc-gh-sghosh avatar Jul 01 '24 03:07 sfc-gh-sghosh

Hello @tekumara ,

Could you try the code in this way, I just tried and I am not getting any compilation error in VScode

`cursor1 = con.cursor(snowflake.connector.cursor.DictCursor)

try:
    r1 = cursor1.execute("select 1").fetchall()
finally:
    cursor1.close()`

sfc-gh-sghosh avatar Jul 02 '24 02:07 sfc-gh-sghosh

That's works but I'm not using execute I'm using execute_string in my codebase to execute multiple sql statements (I simplified it for the MRE above)

tekumara avatar Jul 02 '24 06:07 tekumara

Hello @tekumara ,

You need to split the statements if using multiple statements

Example `main_cursor = con.cursor(snowflake.connector.cursor.DictCursor) print("cursor created")

sql_script = """
CREATE OR REPLACE TABLE my_table (id INT, name STRING);
INSERT INTO my_table VALUES (1, 'Alice'), (2, 'Bob');
SELECT * FROM my_table;
"""

# Split the script into individual statements
sql_statements = sql_script.strip().split(';')


for statement in sql_statements:
    if statement.strip():  # Only execute non-empty statements
        main_cursor.execute(statement)
        try:
            results_cursor = main_cursor.fetchall()
            for row in results_cursor:
                print(row)
        except snowflake.connector.errors.ProgrammingError:
            
            pass

main_cursor.close() con.close()`

Output: cursor created {'status': 'Table MY_TABLE successfully created.'} {'number of rows inserted': 2} {'ID': 1, 'NAME': 'Alice'} {'ID': 2, 'NAME': 'Bob'}

Regards, Sujan

sfc-gh-sghosh avatar Jul 23 '24 04:07 sfc-gh-sghosh

Isn't that what execute_string does for me?

tekumara avatar Jul 23 '24 09:07 tekumara

Hello @tekumara ,

Sorry for the dealy. As of now, for execute_string, you have to split the statements, as recommended above.

Example: from snowflake.connector.util_text import split_statements

def execute_file(filename, conn): with open(filename,'r') as f: text = f.read() try: for stmt in split_statements(StringIO(text)): conn.execute_string(stmt) except: print("Execution aborted")

sfc-gh-sghosh avatar Dec 20 '24 04:12 sfc-gh-sghosh

Ok but I want a DictCursor and I don't really want to roll my own loop, when I could just use execute_string

ie: I want to use

conn.execute_string("select 1",cursor_class=snowflake.connector.cursor.DictCursor)

But I get this type error from pylance/vscode:

Argument of type "type[DictCursor]" cannot be assigned to parameter "cursor_class" of type "SnowflakeCursor" in function "execute_string"
  Type "type[DictCursor]" is not assignable to type "SnowflakeCursor"

Can the type annotations be fixed to prevent this error?

tekumara avatar Dec 20 '24 10:12 tekumara

Hello @tekumara ,

Thanks for the update. At present with execute_string, you can only use cursor_class: SnowflakeCursor = SnowflakeCursor.

So, as a workaround, you can try below code to avoid above error, but you have to split the statements. later you can convert the rows to dictionary format as below ` sql_script = """ CREATE OR REPLACE TABLE my_table (id INT, name STRING); INSERT INTO my_table VALUES (1, 'Alice'), (2, 'Bob'); SELECT * FROM my_table; """

with snowflake.connector.connect(**connection_params) as conn:

for result in conn.execute_string(sql_script):
    try:
       
        rows = result.fetchall()  # Directly fetch rows from the result (cursor)
        if rows:
            # Convert rows to dictionary format
            for row in rows:
                print({desc[0]: value for desc, value in zip(result.description, row)})
    except snowflake.connector.errors.ProgrammingError:
                 pass`

Regards, Sujan

sfc-gh-sghosh avatar Dec 23 '24 05:12 sfc-gh-sghosh

At present with execute_string, you can only use cursor_class: SnowflakeCursor = SnowflakeCursor.

Using cursor_class=snowflake.connector.cursor.DictCursor with execute_string works for me, ie: fetchall() on the result returns a list of dict.

But the type hints are incorrect - they produce a type error but shouldn't.

tekumara avatar Dec 23 '24 11:12 tekumara

Hello @tekumara ,

With the python connector, which you used 3.11.0, and with the latest 3.12.4, there is an error when using it, so it should throw an error for your application.

Using cursor_class=snowflake.connector.cursor.DictCursor with execute_string An error occurred: 'DictCursor' object has no attribute 'execute_string'

So please use the workaround mentioned in previous post.

Regards, Sujan

sfc-gh-sghosh avatar Jan 02 '25 06:01 sfc-gh-sghosh

snowflake-connector-python 3.12.4 does not throw an error for me, eg:

*_, dcur = conn.execute_string("select 1",cursor_class=snowflake.connector.cursor.DictCursor)
dcur.fetchall()
[{'1': 1}]

But I still get the incorrect type error in vscode/pyright.

tekumara avatar Jan 07 '25 08:01 tekumara