CRUD API is not friendly to AWS S3 object keys with path delimiters

[blake-regalia]

For example, /files/{id}/children -- S3 object keys commonly use the / path delimiter, meaning that the command ('children' in this case) coming after the parameter ({id} in this case) makes for a messy request path string. IMO, this is a good indicator that the parameter should be the last fragment of the path string.

Besides, wouldn't it make more sense to combine /files and /files/{id}/children into one command, namely /list/{id} where an empty id means list root?

[kvolkovich-sc]

Hello, @blake-regalia!

It make sense. If current API design isn't suitable for some use-cases, we can think together how to redesign it.

If you have time, you make a proposal how to change other routes too.

http://opuscapita-filemanager-demo-master.azurewebsites.net/api/docs/#

I'll take a look a little later.

[kvolkovich-sc]

server-nodejs has been developed to work with local file system.

/ character in file name is forbidden for most of Linux and Windows file systems.

Another option is just develop direct S3 connector.

I think it's a better option than break current server-nodejs API.

As a bonus, S3-specific connector can allow to handle more S3-specific features.

[kvolkovich-sc]

@blake-regalia, do you use something like https://github.com/s3fs-fuse/s3fs-fuse to map S3 to local FS?

I didn't tried it, but its interesting how it handle such object keys if it POSIX compatible.

If it map them to hierarchy structure, I think it can be solution for it if you don't use docker.

As I know, its not possible to use FUSE inside unpriveleged docker container.

[blake-regalia]

@kvolkovich-sc Thanks for the response. I am all for coding an S3 connector, and its great that the project was set up this way so that this component can be modular.

/ character in file name is forbidden for most of Linux and Windows file systems.

Yes -- just to clarify, an S3 bucket is simply a Map<string key, file value> where key is commonly a string that includes / characters to simulate a hierarchical structure. See here. For all intents and purposes, it's the same for the server to treat these strings as paths that are located on some remote filesystem that's connected via third-party HTTP API.

I think it's a better option than break current server-nodejs API.

I suppose what you're saying is that an S3 connector would have its own separate API. That makes sense too, but I also think it would be nice if the APIs were consistent with each other, maybe even if they inherited from some mutual base API?

[kvolkovich-sc]

If we are talking about client-react API, its just an object with implements minimal interface:

{
  getResourceName: ...
  getResourceById: ({ id }) => resource`
  getChildrenForId: ...
  getParentsForId: ... (should be renamed to getAncestorsForId)
  getParentIdForResource: ...
}

Maybe I forgot some.

These methods allows us to navigate over resources hierarchy and display.

All other options - delete, rename, upload, download are additional capabilities.

They can be specific for each connector and can be available as toolbar buttons or context menu.

S3 connector can implement oauth (something like implemented for Google Drive).

We can start from implementing basic iteraction.

• navigate
• upload
• download
• rename
• delete

After basic functional we can move forward.

• search?
• copy?
• move?
• preview?

I think, its possible to do using S3 REST API

@blake-regalia, what do you think? Do you have time to participate on it?

I'm in no hurry, but I would be interested in playing with AWS an improve Filemanager.

[blake-regalia]

I suppose what you're saying is that an S3 connector would have its own separate API.

If we are talking about client-react API...

Ah, okay -- I think I see the confusion. You're thinking of an S3 connector as a connection from the client directly to an S3 bucket, as the phrase implies :)

However, I was referring to the server HTTP API. Our use-case requires the server mediate interaction between the client and S3 bucket. So the diagram looks like:

[client] <--> [server] <--> [S3]

So I suppose the component to consider in this case is actually server middleware that connects to S3 and exposes the Filemanager HTTP API. This is why I originally opened the issue addressing the HTTP API and its relation to S3 object keys.

---

As for a direct client to S3 connector, s3commander does something similar right now, although there is some concern about keeping the secret key in the browser.

[kvolkovich-sc]

It seemed to me that we are talking about a different things.

Now I understand what you want :)

I'll reproduce your use-case and report the results.

[stephenstubbs]

This project looks brilliant. I am interested in this too. For my use case a direct connector would be fine. How would you suggest I get started doing this? If it works I will open a pr. Otherwise let me know if perhaps I should maybe look at customising the server component but I would rather keep it in the browser if possible.

[kvolkovich-sc]

@sstubbs, thanks!)

It would be great if you start an implementation.

Direct connector can be a good starting point.

Then we can extend it via implementing some simple S3 proxy server as additional connector configuration option.