iceberg-python

iceberg-python copied to clipboard

Support writes with TimeTransforms. TimeTransforms are can be supported natively within pyarrow as pyarrow compute functions, without requiring conversions back and forth between Arrow and Python data types.

sungwy

Add Snowflake catalog

5

- defined reading Iceberg tables using the snowflake catalog. - snowflake catalog is pretty much read only, so adding primarily read only ops. - refer snowflake iceberg sdk read guide:...

prabodh1194

`parquet_path_to_id_mapping` generates incorrect path for List types

6

### Apache Iceberg version main (development) ### Please describe the bug 🐞 When using the `add_files` table api, the parquet metadata needs to be read and a mapping of `Dict[str,...

cgbur

Add support for Python 3.12

2

### Feature Request / Improvement Python 3.12 has been prereleased, and it would be good to support it: https://www.python.org/downloads/ Migrated from https://github.com/apache/iceberg/issues/8499

Fokko

Support `register_table` in GlueCatalog

2

### Feature Request / Improvement `register_table` is currently the only Catalog method not implemented https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/glue.py#L407 Also the docs are misleading because this method is shown in the docs and doesn’t...

rmn36

Support Snapshot Expiration Operation

10

### Feature Request / Improvement Support Maintenance operations on PyIceberg: https://iceberg.apache.org/docs/1.4.0/maintenance All operations except for data file compaction are metadata-only or file system operations, so supporting them on PyIceberg may...

sungwy

Build: Bump pytest-checkdocs from 2.10.1 to 2.13.0

2

Bumps [pytest-checkdocs](https://github.com/jaraco/pytest-checkdocs) from 2.10.1 to 2.13.0. Changelog Sourced from pytest-checkdocs's changelog. v2.13.0 Features Updated collection to prefer 'file_path' over 'path'. (#23) v2.12.0 Features Rely on jaraco.packaging.metadata for loading metadata. (#19)...

dependabot[bot]

ValueError: Mismatch in fields: ?

7

### Question I am just appending to the same table using the same data, ``` try: catalog.create_table("aemo.scada",schema=df.schema) except: catalog.load_table("aemo.scada").append(df) ``` but I am getting this error

djouallah

make to_duckdb accept multiple tables

2

### Feature Request / Improvement currently it seems `con =catalog.load_table('aemo.scada').scan().to_duckdb(table_name="scada") ` accept only 1 table, it will be nice to have something like `con =catalog.load_namespace('aemo').scan().to_duckdb()` which will expose the whole...

djouallah

[feature request] Improve integration test reliance on docker

6

### Feature Request / Improvement It would be nice to eliminate the `sleep 10` and instead use health checks for the underlying containers. https://github.com/apache/iceberg-python/blob/188e2c4d63f294fc55351916fb8d62a5a4401d3d/Makefile#L43 The sleep here is a proxy...

kevinjqliu