trino
trino copied to clipboard
Make region in S3 native file system optional
In some S3-compatible object storage systems the concept of a region as in S3 does not exist. One such example is MinIO. Currently users are forced to provide a fake value for region with these systems.
The desire is to make the region parameter optional. This could be done on the generic S3 level or potentially looking at creating separate levels for each "s3-compatible" system.
Assigning to @electrum as the file system lead.
The SDK v2 used to require region, but it looks like they have a cross region feature now: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/s3-cross-region.html
But even if the SDK requires it .. we could shadow this in Trino and fill it in with some random value if minio is used, right?
Or we could just have a random and invalid default value. Then its essentially optional.. problem of course is that it is then quietly invalid for s3 users unless they fill it to correct the default.
Do we want to close this @electrum ?
Region is required by the AWS SDK as it is part of the authorization header. Since we don't know if the storage system will validate the region, my thought is we should keep it as a requirement.