speckle-server icon indicating copy to clipboard operation
speckle-server copied to clipboard

Stream Blob Storage API

Open didimitrie opened this issue 2 years ago • 1 comments

Usecases

There's several things in the works that need an API to store blobs.

  1. In object pointers to blobs (read: texture support, Revit family files) (stream bound for ACL checks, just like objects)
  2. Comment attachments (read: ability to upload a pdf or image) (stream bound for ACL checks, just like objects)

There's several things that could be grandfathered in to use this API:

  1. File Uploads (stream bound for ACL checks, just like objects)
  2. Previews (stream bound for ACL checks, just like objects) ~~4) Profile pictures (optional)~~

Ecosystem discussions needed

How will this implementation look like from the SDKs side, specifically .NET, Py & related connectors?

Receiving:

  • How does texture support assignment work in Rhino, Blender, Unreal, Unity?
  • How does family loading work in Revit?
  • How will the Viewer load textures for materials?

Sending (less difficult):

  • how does sending a texture look like from .NET & Py?
  • how does sending a RFA file look like?
    • would we want to provide all uploaded revit families as a library on a stream? (suspect this is a common usecase)
    • does this require us to add extra metadata around each file upload?
  • OBJ file upload: materials can have textures! (reminder)

Prelim API Spec Braindump

NOTE: The file uploads module (see here) can be decoupled from its original file-processing goal :)

POST /api/v1/stream/:streamId/blob -> straightforward multipart file upload GET /api/v1/stream/:streamId/blob/:hash -> straightforward file download

This is a s0 :)

I was here :)

didimitrie avatar May 17 '22 21:05 didimitrie

The preview service blob storing capability is currently not possible, since the blob creation api needs authZ.

We could:

  • replicate the file import service, where a temp token is created in the users name. This is quick and dirt and introduces more tangling between services and DBs
  • move the blob storage to a separate service with the server being an API gateway, and use the blob service api without authz.

gjedlicska avatar Jun 17 '22 08:06 gjedlicska