Storage API

[olanod]

A filesystem API as the only means to retain state in an application can be quite limiting, specially if the host platform doesn't have the concept of a filesystem, this can result in a bunch of glue code in the host to emulate a file system and for applications that have simple storage needs it also means bundling more code to adapt to the filesystem interface.

Since WASI isn't supposed to stay as a low level POSIX-like collection of interfaces but also encourages the definition of higher level APIs that can serve specific(but still widely used) use cases, I think a simple storage API can be a great addition.
So how should said API look like? 🤷🏽 That's up for discussion! but here are some ideas and I'll be thrilled to hear some more thoughts ...

• Async. Not sure what's the current status of async calls in WASI but I think this one should be async from the start.
• Simplicity. We want to be high level but not too much to force people into a specific kind of design pattern like an object store or relational DB. Simple Key Value storage of plain bytes should be a good start?
• Isolation. If it's not obvious, like the file system, WASM modules shouldn't be able to access other module's storage.
• Namespaces. Or the lack of it? instead of being able to create and open different DBs perhaps a module just gets a default namespace that is already "open", the user anyway already showed her intention of using the storage by enabling that capability. - - Prefixes. To aid in lack of name spacing, we can make it easy to retrieve keys which follow a common prefix.
• Transactional. I think a simple transactional system can still fit in the design of this simple API. It's my understanding that something like a callback is possible which the host can use to revert changes if it returns an error code or something.

[linclark]

Since different use cases for storage have different requirements, WASI is likely to have multiple Storage APIs. For example, Piotr Sikora has already proposed including a KV store API as part of the work needed for HTTP proxy APIs, and others have talked about relational DBs and GraphQL. So I don't know how much we can generalize in a single issue, but I'll fill in some of the background from previous discussions here.

• Async. Discussions around async are already in progress. You can see the notes from the Proxy-Wasm update and the follow up Supporting alternative ABIs for blocking function calls for more on this.
• Simplicity. As explained above, we will have different APIs for different use cases.
• Isolation. This is a key principle that is well-established in the WASI SG, so reviewers of APIs already use this lens when reviewing.
• Namespaces. We expect modules to be passed handles as capabilities rather than creating connections dynamically. This is consistent with the capability model used elsewhere. It does not require namespacing.
• Prefixes. See above.
• Transactional. I expect this would be a design decision each API would make independently.