iceberg-python

iceberg-python copied to clipboard

Closes #1928 # Rationale for this change Add `validate_deleted_data_files` which depends on #1935 # Are these changes tested? In progress... # References Java `deletedDataFiles` impl: https://github.com/apache/iceberg/blob/3a29199e73f2e9ae0f8f92a1a0732a338c66aa0d/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java#L678 Java `ManifestGroup.entries` impl: https://github.com/apache/iceberg/blob/3a29199e73f2e9ae0f8f92a1a0732a338c66aa0d/core/src/main/java/org/apache/iceberg/ManifestGroup.java#L242

jayceslesar

### Apache Iceberg version None ### Please describe the bug 🐞 Hey, I have a large arrow table that I want to append to a partitioned iceberg table. I'm working...

dor-bernstein

Support reading initial-defaults

5

Fokko

Add support for pyarrow DurationType

4

### Feature Request / Improvement Currently a pa.Schema with a pa.DurationType can't be converted to an iceberg schema. I think it should be treated the same way as a `pa.Time64Type`...

0x26res

table.scan(row_filter="x IN (0, 1)") does not include the values for which x=0 when x is a DoubleType and a partition column

3

### Apache Iceberg version 0.9.0 (latest release) ### Please describe the bug 🐞 Hi, thanks for writing `pyiceberg`. The bug is pretty much described in the title: `table.scan(row_filter="x IN (0,...

ypsah

Include `initial-default` and `write-default` in repr

3

### Feature Request / Improvement Right now we don't show the `initial-default` and `write-default` in the `repr` of the `NestedField`. This is because we use it internally to deserialize the...

Fokko

Move implementation of upsert from Table to Transaction

6

# Rationale for this change Previously, the **upsert** functionality was implemented at the **table** level, which meant it always initiated a new **Transaction**. This change moves the **upsert** implementation to...

koenvo

Request: Support for Atomic Operations (upsert, delete) in Transaction

1

### Question Hello, I am an avid user of pyiceberg. I would like methods such as upsert and delete to operate atomically, but it seems that these methods are not...

jeongmindo-onepredict

feat: validate snapshot write compatibility

4

## Description - This PR checks snapshot write compatibility and validates no conflicting concurrent operations have been written that can clash. - Added Snapshot util file that implements `ancestors_between` and...

kaushiksrini

Hive metastore 4.0.1 remove deprecated thrift APIs

7

### Apache Iceberg version 0.7.1 ### Please describe the bug 🐞 Starting at version 4.0.1, Hive metastore removed deprecated thrift APIs that py-iceberg is currently using. When trying to create...

mattheusv