iceberg-python

iceberg-python copied to clipboard

Fix Azure-incompatible file paths in PyArrowFile

2

# Rationale for this change Starting from version 20, Pyarrow has support for Azure filesystems. Azure table locations are typically of this format: "abfss://@..core.windows.net///". When creating a PyArrowFile, we simply...

NikitaMatskevich

Writing to a table with an optional map field fails if data is missing map field

2

### Question I am not sure if this is a bug or things working as expected, but when writing to a table with an optional map field, if the input...

dbnl-renaud

Add anon property to fsspec adls file io config to ease usage of DefaultCredential pipeline

2

# Rationale for this change We are using default credential pipeline to get access to Azure (more concretely, [managed identities](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview)). We found out that fsspec library [only allows it if...

NikitaMatskevich

Allow Arrow Capsule Interface

3

### Apache Iceberg version 0.10.0 (latest release) ### Please describe the bug 🐞 Due to how `iceberg-python` does checks in certain places, I can't use libraries such as [`arro3`](https://github.com/kylebarron/arro3) or...

DisturbedOcean

# Rationale for this change My expectation for `to_record_batches` was that it would yield batches and not materialize an entire parquet file in memory, but it looks like the current...

tom-at-rewbi

Make And expression JSON serializable using Pydantic

9

#2518 This PR addresses issue by making the `And` expression in pyiceberg.expressions fully JSON serializable using Pydantic. - Added a unit test to verify correct JSON serialization of the And...

Aniketsy

[infra] explore using uv for building wheels

2

### Feature Request / Improvement Context: https://github.com/apache/iceberg-python/pull/1599#discussion_r1938854065 Example: https://github.com/apache/iceberg-python/compare/main...kevinjqliu:iceberg-python:kevinjqliu/build-with-uv ``` CIBW_BUILD_FRONTEND: "build[uv]" ``` https://cibuildwheel.pypa.io/en/stable/options/#build-frontend

kevinjqliu

Error when filtering by UUID in table scan

8

### Apache Iceberg version main (development) ### Please describe the bug 🐞 **Problem** Getting a `pyarrow.lib.ArrowNotImplementedError: Function 'equal' has no kernel matching input types (extension, extension)` when trying to scan...

sevakva

### Apache Iceberg version 0.10.0 (latest release) ### Please describe the bug 🐞 I'm receiving the following stack trace when trying to upsert 5,000 records in batches. I've tried rolling...

nHurD

Draft: Add python 3.13 compatibility

3

# Rationale for this change Updated Ray to 2.45 to allow compatibility with python 3.13 Added filter for new sqlite3 “unclosed database” warnings in Python 3.13. New Python 3.13 warnings...

gregunz