libsql icon indicating copy to clipboard operation
libsql copied to clipboard

Define libSQL<->Wasm ABI for type translation

Open psarna opened this issue 1 year ago • 1 comments

In order to add dev-friendly support for running WebAssembly functions within libSQL, we need to specify the format in which types are passed between the two environments. For instance, when a WebAssembly function is to be called on columns of types integer, real, text, blob, there needs to be a guideline on how the header of this WebAssembly function should look like.

WebAssembly has a narrow set of supported types (https://webassembly.github.io/spec/core/syntax/types.html), and there are a few fundamental questions:

  • Should we try to use native Wasm types for integers and floats?
  • Should we encode strings with trailing \0 or with their size encoded in the prefix?
  • etc.

The most reasonable way seems to be to follow another well defined ABI - the C one: https://github.com/WebAssembly/tool-conventions/blob/main/BasicCABI.md#function-signatures

In particular, it specifies that all non-trivial types, like structs, are passed indirectly, via a pointer. I suggest we follow this practice and decide that all parameters of the WebAssembly function are expected to be simply pointers to the sqlite3_value struct. This structure also stores its type information, so the exact type can be inferred and validated within a WebAssembly function.

Ref: https://github.com/libsql/libsql/blob/aafe1d5457936a05bca5d312e2fd25f32afeb20b/src/vdbeInt.h#L208-L231

With such a specification, helper libraries (https://github.com/libsql/libsql/issues/15) could take care of translating and validating the types, while the users have a clear ABI to follow when implementing user-defined functions which work on libSQL column types.

psarna avatar Oct 05 '22 08:10 psarna

After more consideration - sqlite3_value supports a quite narrow set of types, and they can be naturally mapped to WebAssembly types: INTEGER <-> i64 DOUBLE <-> f64 TEXT, BLOB, NULL <-> a pointer (i32)

Blobs would likely either require one extra layer of indirection, passing a single blob parameter as two WebAssembly parameters - pointer and size, or encoding its size in the first 4 bytes of data. Extra layer of indirection sounds more in line with the C ABI mentioned above. Then, a blob would be passed as a pointer to a structure {p, s}, where p is a pointer to blob's actual data and s is the blob's size.

psarna avatar Oct 07 '22 13:10 psarna