trino-python-client
trino-python-client copied to clipboard
Empty results on large queries when using fetchall()
Hello there.
I've found that these two examples will not always return the same thing on large queries:
- fetchall
cur.execute(large_query)
rows = cur.fetchall()
print(rows)
- iterator
for r in cur.execute(large_query):
print(r)
The fetchall
approach ends up returning an empty list every time I use it when running large queries, but the iterator
approach prints the result successfully
Does anyone know why this may be happening?
I don't know how I should supply a working example to debug this problem, as my queries all contain domain logic of my company. Nevertheless, I can say that I'm connecting around 8 Postgres databases with 100k rows per table and equal schemas, and I union the tables from all databases together with UNION views, like this:
CREATE OR REPLACE VIEW hive.default."table_unioned_view" AS
SELECT 'database1' AS database, "other_column" FROM database1.public.table
UNION ALL
SELECT 'database2' AS database, "other_column" FROM database2.public.table
UNION ALL
SELECT 'database3' AS database, "other_column" FROM database3.public.table;
And the big query that I'm running uses these type of views
Would you like to try to come up with a test case reproducing the problem? See https://github.com/prestosql/presto-python-client/blob/a7438556213bcf954f8a7a5063c7823bbb177edb/integration_tests/test_dbapi.py#L59-L68 for some example test using dockerized Presto