iceberg-python icon indicating copy to clipboard operation
iceberg-python copied to clipboard

Add flag to allow disabling creation of catalog tables

Open isc-psulin opened this issue 1 year ago • 4 comments

Remove unnecessary _ensure_tables_exist method as this is already the default behavior of Metadata.create_all()

isc-psulin avatar Sep 10 '24 13:09 isc-psulin

@isc-patrick do you know how it will check if the table exists? I'm asking since a CREATE TABLE IF NOT EXISTS ... might require CREATE TABLE permissions.

Fokko avatar Sep 10 '24 20:09 Fokko

I can certainly add tests, but that is really testing the Metadata.create_all() function in SQLAlchemy and not pyiceberg code. I think that CREATE TABLE IF NOT EXISTS requires same permissions as CREATE TABLE. How this is executed will be determined by the Dialect.

isc-psulin avatar Sep 11 '24 18:09 isc-psulin

please do! we want to ensure that this change does not break new and existing DB integrations.

kevinjqliu avatar Sep 11 '24 19:09 kevinjqliu

It does not look like CREATE TABLE IF NOT EXISTS is what is used. It is specific to each dialect and how they implement the has_table method, but the ones I looked at use metadata on the schema to determine the existence of the table.

isc-psulin avatar Sep 11 '24 19:09 isc-psulin

Thank you again for working on this PR @isc-patrick !

sungwy avatar Oct 18 '24 00:10 sungwy