Multipart file downloads PR#332 fails with access denied error on requester pays bucket

[woodcockr]

• Async AWS SDK for Python version: 13.x
• Python version: 3.12
• Operating System: Ubuntu 24.04

Description

Download a file over s3 from a public requester pays bucket.

aioboto3 13.x Throw ClientError exception with Access Denied error. Normal download using boto3 (no async) works as expected. aioboto3 <13.0 works as expected.

What I Did

import aiofiles
import boto3
import aioboto3
import asyncio

s3 = boto3.client('s3')

async def download_file():
  session = aioboto3.Session()
  async with session.client('s3') as s3:
    async with aiofiles.open('test_file.tif', 'wb') as data:
      await s3.download_fileobj(
        Bucket='usgs-landsat',
        Key='collection02/level-1/standard/oli-tirs/2023/090/079/LC08_L1TP_090079_20230124_20230207_02_T1/LC08_L1TP_090079_20230124_20230207_02_T1_B6.TIF',
        Fileobj=data,
        ExtraArgs={"RequestPayer": "requester"}
      )

def download_file_sync():
  with open('test_file_sync.tif', 'wb') as data:
    s3.download_fileobj(
      Bucket='usgs-landsat',
      Key='collection02/level-1/standard/oli-tirs/2023/090/079/LC08_L1TP_090079_20230124_20230207_02_T1/LC08_L1TP_090079_20230124_20230207_02_T1_B6.TIF',
      Fileobj=data,
      ExtraArgs={"RequestPayer": "requester"}
    )

# To run the sync function
download_file_sync()

# To run the async function
asyncio.run(download_file())

I'm not familiar with the aioboto3 code base but wondered if this line was missing some ExtraArgs, to add the RequestPayer='requester' get_object appears to require?