sgr icon indicating copy to clipboard operation
sgr copied to clipboard

Support for Azure blob store as well as S3

Open dazzag24 opened this issue 4 years ago • 2 comments

Hi,

Congratulations on a awesome product. I'm looking forward to experimenting with it some more. Currently you have S3 support, Any plans for Azure blob support?

Thanks

Darren

dazzag24 avatar Jul 01 '20 15:07 dazzag24

Thanks and congratulations on being the first issue in the repo!

It's definitely doable. The architecture we have in mind for external storage is that you have two Splitgraph engines, one acting as a client and one acting as the "registry". The registry stores the actual access credentials (e.g. S3 access/secret key) and delegates access to Splitgraph objects to the client when it needs to push/pull data (via S3 pre-signed URLs or Azure shared access signatures).

Does this work for your use case?

mildbyte avatar Jul 02 '20 07:07 mildbyte

I think that would work.

dazzag24 avatar Jul 11 '20 19:07 dazzag24