iceberg-python icon indicating copy to clipboard operation
iceberg-python copied to clipboard

Snowflke can't read tables generated by pyicberg ?

Open djouallah opened this issue 1 year ago • 3 comments

Question

I keep getting this error ?

Failed to read from Iceberg file 'abfss://xxxxxx.dfs.core.windows.net/data/iceberg_dwh/scada/metadata/snap-2728627078701324745-0-7c1d442e-7321-46f8-aa06-5d5f94cde607.avro': Failed to open file: abfss://xxxxxx.dfs.core.windows.net/data/iceberg_dwh/scada/metadata/snap-2728627078701324745-0-7c1d442e-7321-46f8-aa06-5d5f94cde607.avro

djouallah avatar May 10 '24 12:05 djouallah

i see, it is an upstream issue https://github.com/apache/iceberg/issues/10127

djouallah avatar May 11 '24 14:05 djouallah

Normally the abfss URI is expected to look like this, according to azure docs:

abfs[s]://<file_system>@<account_name>.dfs.core.windows.net/<path>/<file_name>

In your example, did your files contain the <file_system> component of the URI? (Or what would be the "container" in Azure Blob syntax)

sfc-gh-dhuo avatar May 13 '24 18:05 sfc-gh-dhuo

the URI is 'abfss://account_name.dfs.core.windows.net/data/iceberg_dwh/scada/metadata/snap-2728627078701324745-0-7c1d442e-7321-46f8-aa06-5d5f94cde607.avro'

djouallah avatar May 14 '24 00:05 djouallah