Experiment: Use Rust code directly as the interface definition.

[rfk]

This is a fun little something I've been messing around with, partly just to learn more about Rust macros and syn, and partly to see if they could make a good replacement for our current use of WebIDL for an interface definition language.

Key ideas:

• Declare the UniFFI component interface using a restricted subset of Rust syntax, directly as part of the code that implements the Rust side of the component. There is no separate .udl file.

• Use a procedural macro to insert the Rust scaffolding directly at crate build time, rather than requiring a separate build.rs setup to generate and include it.

• When generating the foreign-language bindings, have uniffi-bindgen parse the Rust source directly in order to find the component interface.

This seems to work surprisingly well so far. If we do decide to go this route, the code here will need a lot more polish before it's ready to land...but it works! And it works in a way that's not too conceptually different from what we're currently doing with a separate .udl file.

---

[EDIT] I'm going top use the PR summary here to keep a list of the key points we'll need to consider and open questions to be resolved:

• How can we support splitting the Rust implementation code out into a manageable set of files, while still allowing UniFFI to see the whole interface at once?
• How can we support features that are not native to Rust but common in other languages, such as default argument values?
• How can we support good error reporting, ideally providing context-appropriate hints on the user's Rust source code?

[rfk]

I've rebased this atop latest main and simplified it a bit in the hope of making it more approachable. In particular:

• I've removed the UDL-generating backend, which was a fun hack but isn't something we'd want to actually land.
• I've removed support for parsing a Rust file with the include_scaffolding! syntax; we now only support the uniffi::declare_interface macro.

I think a great next step here for interested folks, could be to try porting more of the examples and/or tests over to use the new macro syntax. Probably this won't work for any but the simplest examples, because of missing features! But it will be an interesting exercise in trying to use the new approach.

[rfk]

Architecturally, I think the parsing-related logic in the interface module is getting a bit out of hand here. If we decide we want to keep both the UDL parsing for b/w compat and the new syn-based parsing, I think we should:

• Keep the base interface module as just defining the structs and traits, without any parsing logic.
• Move the weedle-based parsing impls into a submodule like interface/parse/udl.
• Move the syn-based parsing impls into a submodule like interface/parse/syn.

This would help ensure that you can look at just one piece of the parsing logic and a time, and also make it easier to eventually delete the deprecated UDL syntax.

[badboy]

Today I took this branch for a test run against my minimal Glean+UniFFI fork to see how far I get. The Glean UDL is here: https://github.com/badboy/glean/blob/13e765da0fccdfb376c995888b4fdc123419f6c1/glean-core/uniffi/src/glean_core.udl

One big issue with the current implmementation is a bug, see this commit: https://github.com/mozilla/uniffi-rs/commit/9a23722f6d5f6db095e0ae466ca104b5b27f6939 tl;dr: It doesn't handle objects yet properly and generates broken code.

I have some more comments:

I don't think the auto-insertion of pub use $namespace::$thing is a good idea. See this commit: https://github.com/mozilla/uniffi-rs/commit/87448dab9f450564ab3ded40f14c3ed29fb1207f

In the current implementation a lot of errors are hidden. That includes some compile errors, import errors, etc., because the macro happens before the Rust compiler sees the code and can compile it. I had to remove the uniffi::declare_interface a couple of times to fix compile errors first, then add it back and fix the remaining uniffi errors.

Cramming everything of the exposed API into a single module is ... not good. Glean's API is huge (General API + 15+ metric types, each with 3-5 methods). It's unwieldy to keep all these structs, functions and the full function implementations in a single file. If it's kept like that we might resort to hacks like concatenating files ahead of the build to keep proper code organization possible.

---

Initially I thought the macro approach can work really well and with this PR up I thought we might be close to a working implementation we could ship. After spending this morning on it I'm not so convinced anymore. Some of this boils down to needing an improved implementation (bug fixes + better error reporting). And that's just engineering effort.

But IMO right now the requirement of having it all in one module is a showstopper for me and I don't have a good idea how we can work around that. Annotate each enum, struct and impl with #[uniffi::interface]? cargo expand the whole codebase, then parse that to find the right things to uniffi-export? (I now understand why wasm-bindgen does it that way)

[rfk]

But IMO right now the requirement of having it all in one module is a showstopper for me and I don't have a good idea how we can work around that.

That's fair. One other data point on this front is the fxa-client crate, which is the crate for which I was using this little experiment in order to autogenerate the UDL from the Rust code. It's a bit different syntax but I was actually parsing the top-level lib.rs to determine the interface:

• https://github.com/mozilla/application-services/blob/main/components/fxa-client/src/lib.rs

In that case, most of the functions/methods are just little stub wrappers around implementations that live elsewhere in the crate. This helped keep the code organization under control while still exposing all the function signatures in the one Rust file, albeit at the cost of some duplication of the function signatures.

It's probably not a pattern we'd want to force on all consumers, though.

[badboy]

This is now fully superseded by the existing macro approach.