athena-federation-python-sdk

athena-federation-python-sdk copied to clipboard

Fix deprecated read_schema in new pyarrow versions

5

barhot

Implement Python UDF

3

UDFs are also a great feature of Query Federation. Per [this example](https://github.com/aws-samples/aws-athena-udfs-textanalytics/blob/main/athena-udfs-textanalytics/src/main/java/com/amazonaws/athena/connectors/textanalytics/TextAnalyticsUDFHandler.java#L947), it looks like it should be fairly straightforward to build a Python implementation using this existing code.

dacort

public ECR requires authentication, to simplify things I suggest a switch to the same [image](https://hub.docker.com/layers/amazon/aws-lambda-python/3.8.2021.12.09.15/images/sha256-93a864ba8623c717b314a37fedd41f189d4b976e76b5f3be750cb34d03d36154) in Docker Hub

danielhaviv

Ideally we can automate releases of PyPI: Good example here: https://www.caktusgroup.com/blog/2021/02/11/automating-pypi-releases/ ```yaml name: Upload Python Package on: release: types: [created] jobs: deploy: runs-on: ubuntu-20.04 steps: - uses: actions/checkout@v2 - uses:...

dacort

While Splits are supported, I haven't yet added support for partitions or predicate pushdown. Upvote this issue to indicate support for either of these features.

dacort

Build with Poetry (#1)

2

* Fix deprecated read_schema in new pyarrow versions * Use poetry * rename project folder * Python CI * lint * make test

drernie

Partition Support

1

> Partitions are not supported, so Athena will not parallelize the query using partitions. The main reason we wanted to use this SDK was to create Athena tables build from...

drernie