pygeoapi icon indicating copy to clipboard operation
pygeoapi copied to clipboard

Support cloud object storage for STAC

Open aripollak opened this issue 2 years ago • 2 comments

Is your feature request related to a problem? Please describe. Currently, STAC support requires files to be accessible via regular filesystem operations, but I'm working on a project where it would make more sense for the files to live in AWS S3.

Describe the solution you'd like Add the ability to retrieve files for STAC via cloud storage, perhaps via something like https://libcloud.apache.org/ or using pystac?

Describe alternatives you've considered

  • It might be possible to emulate a real filesystem by using something like https://github.com/kahing/goofys, but that requires FUSE support, which isn't currently possible on AWS ECS Fargate.

aripollak avatar Jul 09 '21 20:07 aripollak

Note that libcloud storage doesn't currently seem to have the ability to walk just a single directory level at a time, so I'd probably go with boto3 for efficiency reasons based on the way the existing FileSystem provider works. Many non-AWS providers already have an S3-compatible API, so supporting those should just be a matter of configuring the hostname.

aripollak avatar Jul 14 '21 13:07 aripollak

Thanks @aripollak, sounds good. Might want to consider abstracting the boto3 support within the provider, so that we can extend at some point if needed (for example, for Azure blob storage, etc.).

tomkralidis avatar Jul 14 '21 14:07 tomkralidis

As per RFC4, this Issue has been inactive for 90 days. In order to manage maintenance burden, it will be automatically closed in 7 days.

github-actions[bot] avatar Mar 10 '24 21:03 github-actions[bot]

As per RFC4, this Issue has been closed due to there being no activity for more than 90 days.

github-actions[bot] avatar Mar 24 '24 03:03 github-actions[bot]