iceberg-python

iceberg-python copied to clipboard

Remove deleted data files with expire_snapshots

4

### Feature Request / Improvement Running an expire snapshots operation will only rewrite the metadata file without the expired snapshots (and refs/ statistics). It does not delete deleted data files...

Anton-Tarazi

Incremental Append Scan

1

### Feature Request / Improvement Related is #240 for Incremental Changelog Scan support. This issue tracks Incremental Append Scan support for PyIceberg. PyIceberg lacks incremental read utilities. It's been asked...

smaheshwar-pltr

# Rationale for this change Improve existing dynamodb catalog. ## Are these changes tested? yes.

ForeverAngry

Check storage-credentials before config in table response

1

Closes #2594 # Rationale for this change According to the [Iceberg specification](https://github.com/apache/iceberg/blob/071d5606bc6199a0be9b3f274ec7fbf111d88821/open-api/rest-catalog-open-api.yaml#L3255), clients must first check whether the respective credentials exist in the `storage-credentials` field before checking the `config` for...

coca-alex

fix: upsert with null values in join columns

1

# Rationale for this change This fixes #2426. The upsert method now supports null values to be passed in the join columns. ## Are these changes tested? Yes, I added...

mdwint

Enhance `catalog.create_table` API to enable creation of table with matching `field_ids` to provided Schema

11

### Feature Request / Improvement Currently, `create_table` takes a `pyiceberg.schema.Schema` or `pyarrow.Schema`. The API ignores the `field_id`s provided in the schema and issues a set of new ones. Not only...

sungwy

Add tests for Schema Evolution on Catalogs

1

# Rationale for this change PyIceberg is missing a variety of Catalog Tests compared to the Java implementation. This adds 3 different tests for schema evolution to help match the...

rambleraptor

Possible read-after-write consistency issue with multiple schema migration steps in Iceberg tables on AWS Glue + S3

1

### Apache Iceberg version Pyiceberg 0.10.0 Pyiceberg-core 0.6.0 ### Please describe the bug 🐞 This may be a hard one to pin down but I noticed that multiple schema migration...

din14970

Upsert fails after adding in a new column, target field not found

2

### Apache Iceberg version 0.10.0 (latest release) ### Please describe the bug 🐞 The upsert works perfectly fine until I needed to add a new field in the table. Add...

saul-data

Compatible File storage provider that do not support signature with trailer

1

### Feature Request / Improvement When use fsspec io-impl, put object to File Storage Provider(FSP) not support - [Signature calculations for trailing headers](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming-trailers.html) - [checking-object-integrity-upload](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html) Put file like snapxxx.avro will...

fcfangcc