uniffi-rs icon indicating copy to clipboard operation
uniffi-rs copied to clipboard

Published 20 hours ago • verified publisher icon mozilla

Allow for arbitrary BindingGenerator

Open SalvatoreT opened this issue 1 year ago • 1 comments

De-couple TargetLanguage from the generate_external_bindings calls. This will allow 3rd-party BindingGenerators to use the methods.

I'm not 100% certain this is the way to go, and I'm very open to feedback. cc @bendk @mhammond

Related to https://github.com/mozilla/uniffi-rs/issues/299

SalvatoreT avatar Feb 11 '24 19:02 SalvatoreT

Updated, @mhammond!

SalvatoreT avatar Mar 04 '24 00:03 SalvatoreT

I'd love to get this in the next release if we can get CI green.

mhammond avatar Mar 18 '24 03:03 mhammond

I'm struggling with this one.

❯ cargo test -p unary-result-alias --test test_generated_bindings
   Compiling serde v1.0.186
   Compiling memchr v2.5.0
   Compiling goblin v0.8.0
   Compiling nom v7.1.3
   Compiling weedle2 v5.0.0 (/Users/sal/Developer/uniffi-rs/weedle2)
   Compiling camino v1.1.6
   Compiling serde_json v1.0.105
   Compiling semver v1.0.18
   Compiling cargo-platform v0.1.3
   Compiling basic-toml v0.1.4
   Compiling toml v0.5.11
   Compiling bincode v1.3.3
   Compiling uniffi_core v0.26.1 (/Users/sal/Developer/uniffi-rs/uniffi_core)
   Compiling askama_derive v0.12.1
   Compiling cargo_metadata v0.15.4
   Compiling uniffi_macros v0.26.1 (/Users/sal/Developer/uniffi-rs/uniffi_macros)
   Compiling uniffi_testing v0.26.1 (/Users/sal/Developer/uniffi-rs/uniffi_testing)
   Compiling uniffi_udl v0.26.1 (/Users/sal/Developer/uniffi-rs/uniffi_udl)
   Compiling askama v0.12.0
   Compiling uniffi_bindgen v0.26.1 (/Users/sal/Developer/uniffi-rs/uniffi_bindgen)
   Compiling uniffi_build v0.26.1 (/Users/sal/Developer/uniffi-rs/uniffi_build)
   Compiling uniffi v0.26.1 (/Users/sal/Developer/uniffi-rs/uniffi)
   Compiling unary-result-alias v0.1.0 (/Users/sal/Developer/uniffi-rs/fixtures/regressions/unary-result-alias)
    Finished test [unoptimized + debuginfo] target(s) in 9.07s
     Running tests/test_generated_bindings.rs (target/debug/deps/test_generated_bindings-4938ae4946b33a5e)

running 3 tests
   Compiling unary-result-alias v0.1.0 (/Users/sal/Developer/uniffi-rs/fixtures/regressions/unary-result-alias)
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
test uniffi_foreign_language_testcase_test_py ... ok
test uniffi_foreign_language_testcase_test_swift ... ok
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:20:16: error: unresolved reference: jna
import com.sun.jna.Library
               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:21:16: error: unresolved reference: jna
import com.sun.jna.IntegerType
               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:22:16: error: unresolved reference: jna
import com.sun.jna.Native
               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:23:16: error: unresolved reference: jna
import com.sun.jna.Pointer
               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:24:16: error: unresolved reference: jna
import com.sun.jna.Structure
               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:25:16: error: unresolved reference: jna
import com.sun.jna.Callback
               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:26:16: error: unresolved reference: jna
import com.sun.jna.ptr.*
               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:38:2: error: unresolved reference: Structure
@Structure.FieldOrder("capacity", "len", "data")
 ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:39:25: error: unresolved reference: Structure
open class RustBuffer : Structure() {
                        ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:44:25: error: unresolved reference: Pointer
    @JvmField var data: Pointer? = null
                        ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:46:34: error: unresolved reference: Structure
    class ByValue: RustBuffer(), Structure.ByValue
                                 ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:47:38: error: unresolved reference: Structure
    class ByReference: RustBuffer(), Structure.ByReference
                                     ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:65:64: error: unresolved reference: Pointer
        internal fun create(capacity: ULong, len: ULong, data: Pointer?): RustBuffer.ByValue {
                                                               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:81:13: error: unresolved reference: it
            it.order(ByteOrder.BIG_ENDIAN)
            ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:91:31: error: unresolved reference: ByReference
class RustBufferByReference : ByReference(16) {
                              ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:97:23: error: unresolved reference: getPointer
        val pointer = getPointer()
                      ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:107:23: error: unresolved reference: getPointer
        val pointer = getPointer()
                      ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:109:15: error: unresolved reference: writeField
        value.writeField("capacity", pointer.getLong(0))
              ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:110:15: error: unresolved reference: writeField
        value.writeField("len", pointer.getLong(8))
              ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:111:15: error: unresolved reference: writeField
        value.writeField("data", pointer.getLong(16))
              ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:123:2: error: unresolved reference: Structure
@Structure.FieldOrder("len", "data")
 ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:124:27: error: unresolved reference: Structure
open class ForeignBytes : Structure() {
                          ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:126:25: error: unresolved reference: Pointer
    @JvmField var data: Pointer? = null
                        ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:128:37: error: unresolved reference: Structure
    class ByValue : ForeignBytes(), Structure.ByValue
                                    ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:167:17: error: unresolved reference: it
                it.order(ByteOrder.BIG_ENDIAN)
                ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:170:18: error: unresolved reference: writeField
            rbuf.writeField("len", bbuf.position().toLong())
                 ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:208:2: error: unresolved reference: Structure
@Structure.FieldOrder("code", "error_buf")
 ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:209:44: error: unresolved reference: Structure
internal open class UniffiRustCallStatus : Structure() {
                                           ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:213:44: error: unresolved reference: Structure
    class ByValue: UniffiRustCallStatus(), Structure.ByValue
                                           ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:350:35: error: unresolved reference: Library
private inline fun <reified Lib : Library> loadIndirect(
                                  ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:353:12: error: unresolved reference: Native
    return Native.load<Lib>(findLibraryName(componentName), Lib::class.java)
           ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:357:67: error: unresolved reference: jna
internal interface UniffiRustFutureContinuationCallback : com.sun.jna.Callback {
                                                                  ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:360:54: error: unresolved reference: jna
internal interface UniffiForeignFutureFree : com.sun.jna.Callback {
                                                     ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:363:58: error: unresolved reference: jna
internal interface UniffiCallbackInterfaceFree : com.sun.jna.Callback {
                                                         ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:366:2: error: unresolved reference: Structure
@Structure.FieldOrder("handle", "free")
 ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:370:5: error: unresolved reference: Structure
) : Structure() {
    ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:374:47: error: unresolved reference: Structure
    ): UniffiForeignFuture(`handle`,`free`,), Structure.ByValue
                                              ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:445:32: error: unresolved reference: Library
internal interface UniffiLib : Library {
                               ^
/Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa/uniffi/unary_result_alias/unary_result_alias.kt:554:8: error: unresolved reference: Pointer
    ): Pointer
       ^
test uniffi_foreign_language_testcase_test_kts ... FAILED

failures:

---- uniffi_foreign_language_testcase_test_kts stdout ----
Creating testing out_dir: /Users/sal/Developer/uniffi-rs/target/tmp/unary-result-alias-52ef0dc54a21b8aa
Error: running `kotlinc` failed


failures:
    uniffi_foreign_language_testcase_test_kts

test result: FAILED. 2 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.18s

SalvatoreT avatar Mar 18 '24 04:03 SalvatoreT

error: unresolved reference: jna

do you have jna.jar in your CLASSPATH? For a local test run you might need to manually download it and set the environment variable.

badboy avatar Mar 18 '24 10:03 badboy

Yeah, I had to download jna-5.14.0.jar and ensure that's in my CLASSPATH. I don't quite understand why my installed Android Studio didn't come with what I needed, but here we are.

mhammond avatar Mar 18 '24 15:03 mhammond

Adding jna-5.14.0.jar to the CLASSPATH does the trick. On to the next thing!

SalvatoreT avatar Mar 18 '24 23:03 SalvatoreT

Okay, I figured out the bug!

Before

There's one combined Config that contains four languages inside its bindings field.

https://github.com/mozilla/uniffi-rs/blob/c6ddceea437423b39eee0e706743ced5237b226f/uniffi_bindgen/src/lib.rs#L469-L473

https://github.com/mozilla/uniffi-rs/blob/6e4170d4dab751027aad662837aa394de8ff99ba/uniffi_bindgen/src/bindings/mod.rs#L92-L102

The uniffi.toml files, group the different fields be the bindings.kotlin, bindings.swift, bindings.ruby, and bindings.python.

https://github.com/mozilla/uniffi-rs/blob/45d0f340f7f16aa082acc49ba01e64ecf7e1cda7/fixtures/docstring-proc-macro/uniffi.toml#L1-L9

After

The combined Config is no more, so the bindings.{language} keys aren't needed anymore.

I was getting an error because the package_name wasn't getting picked up. The following change fixed that.

--- a/examples/arithmetic/uniffi.toml
+++ b/examples/arithmetic/uniffi.toml
@@ -1,2 +1 @@
-[bindings.kotlin]
 package_name = "org.mozilla.uniffi.example.arithmetic"

Problem

I can go through almost all of the uniffi.toml files and remove bindings.{language}, and it works great. The only file that's a bit tricky is the examples/custom-types/uniffi.toml file which has three different custom_types.Url instances.

https://github.com/mozilla/uniffi-rs/blob/55139d60715f277773c9fd0ec2e94d6e0b9cb637/examples/custom-types/uniffi.toml#L1-L8

https://github.com/mozilla/uniffi-rs/blob/55139d60715f277773c9fd0ec2e94d6e0b9cb637/examples/custom-types/uniffi.toml#L40-L47

https://github.com/mozilla/uniffi-rs/blob/55139d60715f277773c9fd0ec2e94d6e0b9cb637/examples/custom-types/uniffi.toml#L49-L57

Possible Solutions

  1. Add back a language key to the various language's Configs.
  2. Dynamically set the uniffi.toml file for each test run.
  3. Rename custom_types to {language}_custom_types
  4. Give up?

What do y'all think?

SalvatoreT avatar Mar 19 '24 23:03 SalvatoreT

To be clear, one option is to change, eg, [bindings.swift.custom_types.Url] to [bindings.custom_types.Url]? If so, I don't think that will fly for various reasons.

Add back a language key to the various language's Configs.

At face value, this sounds like "restore things to how they worked before", so IMO is a bit of a no-brainer here?

mhammond avatar Mar 20 '24 13:03 mhammond

I added back the usage of BindingGeneratorDefault, so the various bindings.{language}.* configurations would work without a major change.

I still want to move try_format_code into the Config which will probably require parsing the uniffi.toml file much earlier and passing it's generated Config in to the #generate_bindings calls. I can do that on a separate PR though.

SalvatoreT avatar Mar 21 '24 17:03 SalvatoreT