Compiling to WASM(For Python Wrapper)

[RAbraham]

Hi, I'm new to Rust so I would appreciate any advice.

My intent is build a Python wrapper over DDLog. For version 1, I wanted to just take the output of the final rust code e.g. ddlog -i playpen.dl -L <ddlog/lib> and call it within Python.

My first thought is to compile it to Wasm. Then I could use python-ext-wasm. For e.g. when the following Rust code is converted to Wasm:

#[no_mangle]
pub extern fn sum(x: i32, y: i32) -> i32 {
    x + y
}

I can do

from wasmer import Instance

wasm_bytes = open('simple.wasm', 'rb').read()
instance = Instance(wasm_bytes)
result = instance.exports.sum(5, 37)

print(result) # 42!

In playpen_ddlog, I tried cargo build --target wasm32-unknown-unknown and cargo build --target wasm32-wasi but it failed saying with error output(elided):

....
error[E0599]: no method named `refresh` found for mutable reference `&mut State<'out, 'prompt>` in the current scope
   --> /Users/rabraham/.cargo/registry/src/github.com-1ecc6299db9ec823/rustyline-1.0.0/src/lib.rs:120:14
    |
120 |         self.refresh(prompt, prompt_size)
    |              ^^^^^^^ method not found in `&mut State<'out, 'prompt>`

error: aborting due to 23 previous errors

Some errors have detailed explanations: E0412, E0425, E0433, E0599.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `rustyline`.
warning: build failed, waiting for other jobs to finish...
error: build failed

• I don't think I'll need rustyline when I'm calling it within a Python program. Is there anyway of building while filtering out such libraries.
• If I do manage to compile to Wasm, I am not sure about the public interface of the program. Is there some file with extern definitions for e.g. which I can refer to, where I see how to run the start, insert or dump, from a Wasm interface?

If I'm on the wrong track, then please feel free to suggest a better way of doing this.

If there is existing work/plans or thoughts on having a Python/Interpreter interface, please let me know.

[ryzhyk]

Hi @RAbraham ,

Python bindings would be a really valuable contribution, thanks for looking into doing it!

I have no experience with WASM, but I think the easiest way to create Python bindings to DDlog is through the C API. After you run cargo with the standard x86 target on the generated Rust project, it will generate a dynamic library that exposes the C API described here: https://github.com/vmware/differential-datalog/blob/master/rust/template/ddlog.h

If you can build a Python wrapper that loads this library and invokes it via the C API, you'll have access to the entire DDlog functionality.

Does this make sense? Leonid

[mihaibudiu]

No one is working on a Python API or an interpreter interface. We welcome any contributions.

Your approach does not seem to work for interpreter: you still need to compile the DDlog program to rust and compile the rust to Web Assembly. The main reason for an interpreter API is faster turnaround with the debug cycle; today the slowest phase is the Rust compilation itself. I suspect that compiling to WASM won't be any faster than compiling to native code, so I don't know if your approach will really solve this particular problem.

If you want just Python bindings compiling to native code may be easier.

[RAbraham]

Hi @RAbraham ,

Python bindings would be a really valuable contribution, thanks for looking into doing it!

I have no experience with WASM, but I think the easiest way to create Python bindings to DDlog is through the C API. After you run cargo with the standard x86 target on the generated Rust project, it will generate a dynamic library that exposes the C API described here: https://github.com/vmware/differential-datalog/blob/master/rust/template/ddlog.h

If you can build a Python wrapper that loads this library and invokes it via the C API, you'll have access to the entire DDlog functionality.

Does this make sense? Leonid

It does :). I'll try it out and get back to you. Thanks

[RAbraham]

No one is working on a Python API or an interpreter interface. We welcome any contributions.

Your approach does not seem to work for interpreter: you still need to compile the DDlog program to rust and compile the rust to Web Assembly. The main reason for an interpreter API is faster turnaround with the debug cycle; today the slowest phase is the Rust compilation itself. I suspect that compiling to WASM won't be any faster than compiling to native code, so I don't know if your approach will really solve this particular problem.

If you want just Python bindings compiling to native code may be easier.

My bad. I shouldn't have combined two questions in one. If I restrict myself, this question is about compiling to Wasm. So, in that case, completely agree with your advice. Thank you.

[flexera-cnixon]

@RAbraham In case you're still interested I have a wip wasm32-wasi port, it only seems to require a couple of tweaks and 1 function.

So far all I've verified is that it compiles, you'll need a nightly toolchain with the wasm32-wasi target then it's just a case of

cargo +nightly wasi build --lib

[ryzhyk]

@flexera-cnixon , this is exciting! Is your end goal to use it in the browser? How does wasm affect compilation time? Is it slower or faster than normal x86 compilation?

I'd be very interested to know if it works, performance, etc., but in any case please do submit a PR when you're done even if it's not in a fully working state, so that we can pick up from where you left.

Thanks!

[c-nixon]

Whoops, used the wrong account for my previous comment 🙂

My current goal is to run on top of wasi (which provides a sandboxed posix-ish API for system resources for the wasm32 target) so I can load the compiled ddlog in any language supported by wasmer

[RAbraham]

@RAbraham In case you're still interested I have a wip wasm32-wasi port, it only seems to require a couple of tweaks and 1 function.

So far all I've verified is that it compiles, you'll need a nightly toolchain with the wasm32-wasi target then it's just a case of

cargo +nightly wasi build --lib

@c-nixon I've temporarily moved on to other ideas but this is indeed very exciting work. Look forward to seeing this work integrated back in DDLog!