aws-sdk-pandas

aws-sdk-pandas copied to clipboard

Timestamps not being saved correctly to arraw dataset

1

My source data is polars, but I don't think that's causing this issue, I'm trying to write a parquet hive to s3 and have been experimenting with `aws-sdk-pandas` My code...

david-waterworth

fix(athena): fixed write_iceberg which could fire a ICEBERG_TOO_MANY_OPEN_PARTITIONS Athena error

10

Feature or Bugfix Bugfix Detail Athena: fixed write_iceberg which could fire a ICEBERG_TOO_MANY_OPEN_PARTITIONS Athena error Relates None Details This is not really a bugfix given that this is a problem...

erwan-simon

Add support for polars (perhaps via narwhals)

1

**Is your feature request related to a problem? Please describe.** I've mostly migrated my python analytics workflow from pandas to `polars`. This means in order to use a lot of...

david-waterworth

Support S3 Access Grants

1

**Is your idea related to a problem? Please describe.** [S3 Access Grants](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-grants.html) are the recommended solution for accessing files in S3 by end users (the target group of AWS SDK...

jornfranke

Ability to add metadata to parquet/orc schemas directly

1

**Is your feature request related to a problem? Please describe.** There is a new requirement to have metadata directly in all our cloud data at my workplace (this is due...

walter9388

chore(deps-dev): bump the development-dependencies group across 1 directory with 7 updates

2

Bumps the development-dependencies group with 7 updates in the / directory: | Package | From | To | | --- | --- | --- | | [poetry](https://github.com/python-poetry/poetry) | `1.8.3` |...

dependabot[bot]

chore(deps): bump the production-dependencies group with 4 updates

2

Bumps the production-dependencies group with 4 updates: [boto3](https://github.com/boto/boto3), [botocore](https://github.com/boto/botocore), [setuptools](https://github.com/pypa/setuptools) and [pyodbc](https://github.com/mkleehammer/pyodbc). Updates `boto3` from 1.35.35 to 1.35.39 Commits 9dd723f Merge branch 'release-1.35.39' dff7abc Bumping version to 1.35.39 b29280c Add...

dependabot[bot]

fix: handle partitions with empty table in read_parquet with dataset=True

10

When reading a set of parquet files with dataset=True, if the first partition is empty the current logic for dtype inference will fail. It ill raise exceptions as follows: pyarrow.lib.ArrowTypeError:...

cournape

Athena to iceberg method not writting data to columns that are new in the schema

4

### Describe the bug I have a table that was created by a glue job. I want to append data to that table using AWS Wrangler. The writting process seems...

lautarortega

Duplicates values in identity column when mixing redshift.copy and redshift.to_sql

2

Hello everyone! We are having an issue with duplicated values in an identity column when executing `redshift.to_sql()` after executing a `redshift.copy()`. When trying to insert more than 1000 records we...

laderjk