Support for S3 buckets

[g2giovanni]

Hello everyone, is there a way to use stac-browser with COG files stored on a private S3 bucket?

[m-mohr]

I don't think so, but I've never worked with private S3 buckets so I can't really tell, unfortunately.

[rowanwins]

Ultimately I suspect this is a problem that will need to be solved outside of the stac-browser client app.

If we limit the problem to the rendering of COGS then I can see two main approaches

1. Use presigned-urls. Currently when the client requests tiles it sends a url like https://myendpoint/cog/tiles/1/2/3?url=https://somewhere.com/mycog.tif. As it stands the Tiling Infrastructure (TiTiler or RadiantTiler) doesn't have access to that COG file. However presigned urls are a way of providing access to secure file via a URL. So the request to the tiler might look something like https://myendpoint/cog/tiles/1/2/3?url=https://somewhere.com/mycog.tif?a-bunch-of-aws-presigned-params.
   You would however need a server with the bucket credentials that could generate the pre-signed url for you, and we'd also need to look at whether TiTiler and RadiantTiler would respect those url parameters. Maybe within the app there could be a config flag for the app that specified an endpoint for retrieving presigned urls, but we leave it up to each deployer to write their own server for generating those urls?

2. Enhance TiTiler or RadiantTiler to better support auth Somehow write the AWS auth process into TiTiler or the RadiantTiler.

Hope that helps

[m-mohr]

Yes, I suspect the same.

One note regarding the tiling: STAC Browser also supports client side COGs (via stac-layer/geotiff.js) and TiTiler or RadiantTiles might not even be involved.

[DanielJDufour]

Hi, all 👋

I'm a big fan of the pre-signed URLs that @rowanwins mentioned. They definitely make things easier with the client-side visualization libraries. I think it's important to note that (last I checked), geotiff.js doesn't provide an easy way to provide authentication headers to the internal metadata and tile requests. We could potentially add it through a PR to geotiff.js.

I think TiTiler (because it uses rasterio/gdal) already supports reading from private or requester pays S3 buckets. You just have to set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environmental variables. I think I tested this a year or so ago accessing NAIP requester pays imagery, but can't recall 100%. Of course with the caveat that this is for buckets that your S3 account has access to. I'm honestly not sure what to do if an external user is "requesting" access to a bucket owned by someone else.

I like @rowanwins idea of having an authentication/signing endpoint.

No solutions, just thought I could provide a little more context :-)

[rowanwins]

I think geotiff.js would work fine with presigned urls as the auth info is sent via url params rather than headers 🤞

[DanielJDufour]

Agreed 👍 I'll keep my fingers crossed, too. haha