aws-sdk-pandas

aws-sdk-pandas copied to clipboard

Sanitization of named parameters in athena.read_sql_query

4

If I am reading the code for `wr.athena.read_sql_query` correctly, it looks like sanitization is not performed on named queries which are resolved client side. I think this would be a...

frenchytheasian

Support AWS DMS Partitions in Method store_parquet_metadata

2

**Is your idea related to a problem? Please describe.** I am trying to create glue tables for data in S3 written by AWS DMS as partitioned parquet files. The problem...

FleischerT

change the name of "mode" argument to awswrangler.s3.to_csv

5

### Describe the bug The method (awswrangler.s3.to_csv) supports a "mode" argument and **pandas_kwargs. The "mode" argument is not passed through to Pandas, but consumed in the awswrangler method, which also...

goleash-4alight

**Is your idea related to a problem? Please describe.** Consider supporting multiple filesystems as an argument to `wr.s3.*` methods. Two in particular: - Arrow filesystem (current distributed default) - Custom...

jaidisido

`DescribeTable` call hidden in `read_items`

3

I'm using [`read_items`](https://aws-sdk-pandas.readthedocs.io/en/stable/stubs/awswrangler.dynamodb.read_items.html) but inside that call is another API call. Can this be avoided? https://github.com/aws/aws-sdk-pandas/blob/6c0f65b6b63b223bec1059ecd037697b068f7e63/awswrangler/dynamodb/_read.py#L617 To: https://github.com/aws/aws-sdk-pandas/blob/6c0f65b6b63b223bec1059ecd037697b068f7e63/awswrangler/dynamodb/_utils.py#L22 To: https://github.com/aws/aws-sdk-pandas/blob/6c0f65b6b63b223bec1059ecd037697b068f7e63/awswrangler/dynamodb/_utils.py#L42 When I enable debug logging (`boto3.set_stream_logger(name='botocore')` and call `table.key_schema` I...

mooreniemi

Add support for Elasticache

3

**Is your feature request related to a problem? Please describe.** A common use case is upload data from S3 to AWS elasticache (Redis/Memcache). It would be great to have this...

kkalyan

From redshift DB, `read_sql_query` distorts values, but `unload` does not

9

### Describe the bug In our private redshift database on AWS, I have a table with a column `probability` of type `numeric(38, 20)`, its values are between 0 and 1....

lampretl

awswrangler.s3.read_parquet() chunked=True 1 dataframe per result

11

Hi there, I have a question regarding the `chunked=true` option in awswrangler.s3.read_parquet(). I'm looking to load parquet files from S3 in the most memory efficient way possible. Our data has...

matthewyakubiw

Possible to write spark dataframes to glue tables in similar fashion as awswrangler.s3.to_parquet

4

If it isn't possible already, it would be nice i we can use spark dataframes to write to glue tables using something similar to wranglers to_parquet method. It works great...

aabid0193

pls add wr.s3.to_pickle

1

**Is your feature request related to a problem? Please describe.** **Describe the solution you'd like** A clear and concise description of what you are expecting. **Describe alternatives you've considered** A...

b-y-f