Support Pandas s3fs

[eden-ohana]

S3FS

S3Fs is a Pythonic file interface to S3. It builds on top of botocore. The most common way to interact with s3 on Pandas.

API

Usage

import pandas as pd

key = "files/books.csv"

books_df = pd.read_csv(
    f"s3://{AWS_S3_BUCKET}/{key}",
    storage_options={
        "key": AWS_ACCESS_KEY_ID,
        "secret": AWS_SECRET_ACCESS_KEY,
        "token": AWS_SESSION_TOKEN,
    },
)