How to adjust `#[frb(external)]` with Trait bound?

[devgony]

Discussed in https://github.com/fzyzcjy/flutter_rust_bridge/discussions/2101

^{Originally posted by devgony June 18, 2024} First of all, Could you please check if i can use automatic third-party scanning for this crate If not, I want to mirror these methods with Trait bounds

#[frb(external)]
impl<T: GStore + GStoreMut> Glue<T> {
    pub fn new(storage: T) -> Self {}
    pub async fn plan<Sql: AsRef<str>>(&mut self, sql: Sql) -> Result<Vec<Statement>> {}
    pub async fn execute_stmt(&mut self, statement: &Statement) -> Result<Payload> {}
    pub async fn execute<Sql: AsRef<str>>(&mut self, sql: Sql) -> Result<Vec<Payload>> {}
}

But it gets

error[E0412]: cannot find type `T` in this scope
  --> pkg/dart/rust/src/api/simple.rs:74:34
   |
74 | impl<T: GStore + GStoreMut> Glue<T> {
   |                                  ^ not found in this scope

error[E0207]: the type parameter `T` is not constrained by the impl trait, self type, or predicates
  --> pkg/dart/rust/src/api/simple.rs:74:6
   |
74 | impl<T: GStore + GStoreMut> Glue<T> {
   |      ^ unconstrained type parameter

Doesn't it support #[frb(external)] with Trait bound? Do you know how to solve it?

---

fzyzcjy 2 hours ago Maintainer 영어로 된 원본 주석 -

I briefly checked https://github.com/gluesql/gluesql/blob/main/pkg/javascript/src/lib.rs, the JS api of that Rust crate to see what it looks like.

Looks like it has only several functions instead of many. Then in that case, I guess manually mirror some methods are quicker. (But if you need to mirror a ton of functions, maybe consider scanning the crate automatically.)

I want to mirror these methods with Trait bounds

Firstly,

fn plan<Sql: AsRef>(&mut self, sql: Sql) may be

fn plan(&mut self, sql: String) Secondly, generics are not supported yet (thus the T error). Thus I personally suggest checking how https://github.com/gluesql/gluesql/blob/main/pkg/javascript/src/lib.rs is done (it has no generics etc) and maybe follow that way.

This doc may be related as well: https://github.com/fzyzcjy/flutter_rust_bridge/blob/feat/12337/website/docs/guides/third-party/manual/wrappers.md

EDIT: Oh I realize you are 3rd contributor to that repo! Then the answer depends on your needs. For example, originally I thought you are an user, then I suggest creating small API that suffices your needs; but if you are package contributor and want to add official Dart/Flutter support, then that would be another story - e.g. whether you want to expose to users a very rich API.

EDIT: I see the PR https://github.com/gluesql/gluesql/pull/1503 code. For things like https://github.com/devgony/gluesql/blob/6d059cb1314a97f87b43bb1246a093d6bcb49f8f/pkg/dart/rust/src/api/value.rs#L12, I think it can be automatically scanned. For complex things like the generic T you mentioned above, I guess there needs to be some manual work (to avoid code duplication, possibly also done with a Rust macro). One way is to firstly try to scan it automatically (and maybe put #[frb(ignore)] on some things), and then manually add other things that cannot be scanned.

I am modifying doc about this, thus please refer to https://github.com/fzyzcjy/flutter_rust_bridge/tree/feat/12337/website/docs/guides/third-party for latest before it is merged.

2 replies 1 new @devgony devgony 1 hour ago Author 영어로 된 원본 주석 -

EDIT: Oh I realize you are 3rd contributor to that repo! Then the answer depends on your needs. For example, originally I thought you are an user, then I suggest creating small API that suffices your needs; but if you are package contributor and want to add official Dart/Flutter support, then that would be another story - e.g. whether you want to expose to users a very rich API.

Even though the JS (or Python) binding of GlueSQL used just simple wrapper, Dart is something different. Cuz it will be the first static typed language we are gonna support. I will make it as a standard of typed language binding.

So as you said, I wanted to expose very rich API. After Dart 3, with the power of switch expression and pattern matching, i thought it would be really cool to use all the types of GlueSQL in Flutter.

EDIT: I see the PR gluesql/gluesql#1503 code. For things like https://github.com/devgony/gluesql/blob/6d059cb1314a97f87b43bb1246a093d6bcb49f8f/pkg/dart/rust/src/api/value.rs#L12, I think it can be automatically scanned.

At first i thought it would be simple to implement with few converting wrapper like that. Later, i tried #[frb(mirror(Error)] and it looks like i have to mirror all the types. Cuz the AST of SQL and enum Error are various in our project and i want flutter users to get rich types well eg)

Thanks to you, I just saw there is #[frb(ignore)]. So i will try automatic scan and ignore first!.

By the way.. When we automatically scan, Can we set workspace name instead of remote crate name? Dart binding will be one of GlueSQL's workspace and I want to sync with the latest main branch even though it is not released yet.

pkg/dart/Cargo.toml

[dependencies] gluesql-core.workspace = true memory-storage.workspace = true @fzyzcjy fzyzcjy 1 hour ago Maintainer 영어로 된 원본 주석 -

I see. Ping me if there are any questions!

Btw, what about converting this discussion to a github issue, since as we discuss further, this UI will quickly fold things except for several comments iirc and thus not very convenient.

When we automatically scan, Can we set workspace name instead of remote crate name?

Try to specify the crate name, and it uses cargo metadata to find that crate, thus it should find the one in you workspace, as long as your cargo.toml dependency points to your workspace.

[devgony]

Try to specify the crate name, and it uses cargo metadata to find that crate, thus it should find the one in you workspace, as long as your cargo.toml dependency points to your workspace.

You meant even if i just put crate name, it will automatically look for the workspace?

# pkg/dart/flutter_rust_bridge.yaml
rust_input: crate::api,gluesql-core,gluesql_memory_storage
dart_output: lib/src/rust

flutter_rust_bridge_codegen generate
cd rust
cargo check

it gets error:

// rust/src/frb_generated.rs
..
impl SseEncode for gluesql_memory_storage::MemoryStorage {
// failed to resolve: use of undeclared crate or module `gluesql_memory_storage`
..

It should be memory_storage::MemoryStorage. Cuz memory_storage is the workspace name which is different with crate name. How can i rename it or specify the workspace name in flutter_rust_bridge.yaml?

[fzyzcjy]

Cuz memory_storage is the workspace name which is different with crate name.

Oops I see. Such rename is not supported yet (since nobody said before and I did not know it), but feel free to PR!

Alternatively, is it possible to rename your names, such that their name matches? (I guess this is not a breaking change since end users see gluesql_memory_storage, but please correct me if I am wrong)

[fzyzcjy]

Btw, since you control the "third party" crate, this may be helpful: https://cjycode.com/flutter_rust_bridge/guides/miscellaneous/attributes#specifying-attributes (I will add a doc soon)

EDIT: Doc preview (just screenshot of my local dev version) - indeed quite simple:

It is on #2100 as well.

[devgony]

Alternatively, is it possible to rename your names, such that their name matches? (I guess this is not a breaking change since end users see gluesql_memory_storage, but please correct me if I am wrong)

As you told, I renamed our workspace names. But I'm also interested in contributing to your project as well. I will try to create PR to implement rename feature within few weeks.

1. By the way, when i tried auto scan, i got this warning.

To use the automatically generated getters of the following fields of opaque types, it is suggested to read https://fzyzcjy.github.io/flutter_rust_bridge/guides/types/arbitrary/rust-auto-opaque/properties to know more details. (Related fields: gluesql_core::ast::Assignment.value, gluesql_core::ast::Assignment.value, gluesql_core::data::schema::SchemaIndex.expr, gluesql_core::data::schema::SchemaIndex.expr, gluesql_core::data::schema::SchemaIndex.created, gluesql_core::data::schema::SchemaIndex.created, gluesql_memory_storage::Item.schema, gluesql_memory_storage::Item.schema, gluesql_memory_storage::Item.rows, gluesql_memory_storage::Item.rows)

To solve this, should i use both auto scan and manual mirror + non_opaque ?

2. And also the generated code had errors like.. it tried to import private module or missed the importing sentence in some part. is there way to solve this ?

[fzyzcjy]

Looking forward to your PR!

when i tried auto scan, i got this warning

Firstly, do you really want those automatic getters/setters? If not, we can ignore it. If yes -

• Will marking that type as a non_opaque be helpful for your case?
• Will use https://cjycode.com/flutter_rust_bridge/guides/miscellaneous/proxy for the auto generated getter be helpful?

And also the generated code had errors like.. it tried to import private module or missed the importing sentence in some part. is there way to solve this ?

Yes!

• For manual import things: use rust_preamble: blahblah in flutter_rust_bridge.yaml
• For error import private module: This is probably a bug in auto scanner - as I say in the doc it is hard to parse all those imports since 3rd party crate can have very fancy imports. Feel free to open a separate issue with reproducible samples (ok for your case maybe just that existing PR?) and I will have a look

[devgony]

https://github.com/gluesql/gluesql/pull/1503/files#diff-e02d5b149934978d9452cc5ea33cabe9320024ad20cd2929d3e0d3aae39ae9f8R15

// pkg/dart/rust/src/frb_generated.rs
use gluesql_core::ast_builder::expr::function::*; // module `expr` is private

This line got error: module expr is private

It looks like it happens when the third party module used pub use

https://github.com/gluesql/gluesql/blob/main/core/src/ast_builder/mod.rs#L79

// core/src/ast_builder/mod.rs
pub use expr::{
    aggregate::{avg, count, max, min, stdev, sum, variance, AggregateNode},
    function,
};

The correct generated code should be

// pkg/dart/rust/src/frb_generated.rs
use gluesql_core::ast_builder::function::*;

[fzyzcjy]

Yes, currently it parses

pub use expr::aggregate::function;
pub use expr::aggregate::*;

But may not parse that complex pub use. But

pub use expr::aggregate::{avg, count, max, min, stdev, sum, variance, AggregateNode};

can be supported (i.e. {}). I will work on it later.

[devgony]

What can be the workaround for now? just ignore the whole module?

[fzyzcjy]

If you do not want it, try to ignore the module. If you want it, try pub use each and every item...

But I may implement it today (not guaranteed; but if you see a PR then I do implement it), so if you are not in a hurry, just wait a bit :)

[fzyzcjy]

I may also support the more complicated form now, i.e.

pub use expr::{
    aggregate::{avg, count, max, min, stdev, sum, variance, AggregateNode},
    function,
};

since the parsing looks similar

[devgony]

https://github.com/gluesql/gluesql/blob/518a95101e8973b1923575eec0e2a441d5387d81/pkg/dart/rust/src/frb_generated.rs#L3050

I found another issue.

// GlueSQL
// core/src/translate/mod.rs
use {
    crate::{ast::Statement, ..},
    sqlparser::ast::{Statement as SqlStatement, ..},
};

pub fn translate(sql_statement: &SqlStatement) -> Result<Statement> 

Cuz GlueSQL has translate layer from sqlparser::ast::Statement to gluesql::ast::Statement, There are few aliasings like sqlparser::ast::{Statement as SqlStatement, ..} But frb looks like does not parse well.

// FRB
// pkg/dart/rust/src/frb_generated.rs
fn wire__gluesql_core__translate__translate_impl(..) {
  ..
  flutter_rust_bridge::for_generated::RustAutoOpaqueInner<SqlStatement> // cannot find type `SqlStatement` in this scope
  ..

To solve it, there should be import and aliasing like this.

// FRB
// pkg/dart/rust/src/frb_generated.rs
use gluesql_core::sqlparser::ast::Statement as SqlStatement;

I can maually add it by rust_preamble, But it would be nicer if it's done automatically

[fzyzcjy]

Yes, aliasing is not supported yet. I may work on that later, but the priority may not be super high, since it may complicate things. Also feel free to PR!

[devgony]

I know lifetime is not fully supported yet But according to this, It seems like my simple enum with lifetime should be supported but it doesn't

// gluesql/core/src/store/function.rs
#[derive(Clone, Debug)]
pub enum ExprNode<'a> {
    Expr(Cow<'a, Expr>),
    SqlExpr(Cow<'a, str>),
    Identifier(Cow<'a, str>),
    ..

// pkg/dart/rust/src/frb_generated.rs
impl SseDecode for ExprNode { // 🦀 implicit elided lifetime not allowed here
    // Codec=Sse (Serialization based), see doc to use other codecs
    fn sse_decode(deserializer: &mut flutter_rust_bridge::for_generated::SseDeserializer) -> Self {
        let mut inner = <RustOpaqueMoi<
            flutter_rust_bridge::for_generated::RustAutoOpaqueInner<ExprNode>,
        >>::sse_decode(deserializer);
        return flutter_rust_bridge::for_generated::rust_auto_opaque_decode_owned(inner);
    }
}

Is it bug? or there is some option i can try? i already tried enable_lifetime: true but did not work

I manually added <'a> and the last issue is macro frb_generated_moi_arc_impl_value which seems like does not support lifetime as well.

// pkg/dart/rust/src/frb_generated.rs
flutter_rust_bridge::frb_generated_moi_arc_impl_value!(
    flutter_rust_bridge::for_generated::RustAutoOpaqueInner<ExprNode>
);

`impl` item signature doesn't match `trait` item signature
expected signature `fn(&'1 <RustAutoOpaqueInner<ExprNode<'3>> as MoiArcValue>::get_pool::POOL) -> &'1 std::sync::RwLock<MoiArcPoolInner<RustAutoOpaqueInner<ExprNode<'3>>>>`
   found signature `fn(&'1 <RustAutoOpaqueInner<ExprNode<'_>> as MoiArcValue>::get_pool::POOL) -> &'1 std::sync::RwLock<MoiArcPoolInner<RustAutoOpaqueInner<ExprNode<'1>>>>`
the lifetime requirements from the `impl` do not correspond to the requirements in the `trait`
verify the lifetime relationships in the `trait` and `impl` between the `self` argument, the other inputs and its output

[fzyzcjy]

Hi, could you please provide a minimal reproducible sample? Then I can check what is happening. Or, just provide full example (is latest code on that PR?), since this one does not need much reduction to debug I guess.

Without more info, I suspect it is because you have some function signature that confuses current frb and make it believe ExprNode should be treated without lifetime.

[devgony]

Yes I just pushed to that PR. you can work on pkg/dart and i'm still using the basic template of rust/src/api/simple.rs https://github.com/gluesql/gluesql/pull/1503/files#

And the issue is at this line https://github.com/gluesql/gluesql/pull/1503/files#diff-421342a550ee55eb0ad9eaa470d13d3967ccf529a9bc8b0c1b7245f332fe8012R3575

[fzyzcjy]

I will have a look at it soon!

[fzyzcjy]

Btw when checkout that branch, I see flutter_rust_bridge.yaml and thus add a bit in doc discussing multiline:

[fzyzcjy]

And btw, since that will be a package instead of app, this may be helpful: https://cjycode.com/flutter_rust_bridge/manual/integrate/cargokit; or https://cjycode.com/flutter_rust_bridge/guides/miscellaneous/pure-dart (but that requires Dart feature that is not on stable yet)

[fzyzcjy]

Hmm, enable_lifetime: true is not in flutter_rust_bridge.yaml yet, thus it is not enabled.

[devgony]

Hmm, enable_lifetime: true is not in flutter_rust_bridge.yaml yet, thus it is not enabled.

I tried and removed that line. cuz even after using that option, the issue was same and other part got worse with more errors..

[fzyzcjy]

Ok, I will try to enable it and see what is going on. Because if disable it, then lifetime is definitely not supported, then we can never compile with those struct-with-lifetimes!

[fzyzcjy]

Btw, a side question: I am curious is there any doc about why/when should someone choose GlueSQL compared with other databases? (e.g. I have heard of MySQL, MongoDB, ... on server side; Hive, Isar, SQLite, Drift, Real, ObjectBox,... on Flutter side; etc etc...) GlueSQL looks interesting to me by glancing at the docs, but it would be great if someone from GlueSQL can explain a bit, since I definitely cannot say I am a database expert!

My personal guess: Firstly, it is in Rust, thus (usually) fast and safe (especially https://github.com/search?q=repo%3Agluesql%2Fgluesql+%22unsafe%22&type=code seems no unsafe code); Secondly, maybe a lot of storages available, making it good for some scenarios.

[fzyzcjy]

Btw, dump_all: true can be enabled in config, and then rust/target/frb_dump will contain a lot of information for each stage. (I may make some doc discussing it later)

[fzyzcjy]

The latest frb has some confusing error (possibly because it now understands more advanced pub use statements thus scan more things?), and I make them more explicit in https://github.com/fzyzcjy/flutter_rust_bridge/pull/2124. To use frb code in master branch or a PR branch, can use:

cargo run --manifest-path /path/to/the/flutter_rust_bridge/frb_codegen/Cargo.toml -- generate

Thus it seems these things needs to be solved (e.g. one error is "struct with unit fields are not supported yet)

I will (temporarily) halt here, but feel free to ping me if there are any issues!

[devgony]

Cuz those documents are quite outdated.. I will manually answer it here.

1. Basically GlueSQL is not a full DB, but rather an SQL library. (Of course it works as full DB with sample storages we already implemented though). With well-designed traits, you can make customizable DB. e.g.) If you want read only DB, just implement few methods of Store trait. If you want full working OLTP read/write DB, implement Store+StoreMut+Index trait..

2. With GlueSQL, you can use ANSI SQL with anything that can read or write. We already support 11 sample storages {composite, csv, indexedDB, json, memory, mongo, parquet, redis, shared-memory, sled, web(local,session)}, but it can be anything more like notion or discord even github! Didn't you feel uncomfortable whenever using github advanced search? You can do complex search or join with just ANSI SQL. Mongo storage(i implemented) is cool on which we can use ANSI SQL with document DB. Storages on web browser(local, session, indexedDB) are also nice.

3. Cuz it supports various storages, it can be used as ETL solution. With composite-storage of GlueSQL, you can migrate anything to anything with ANSI SQL, eg) from CSV to Mongo, Mongo to some RDB.

4. The most important part with Flutter. Among 11 sample storages, most of storages are embedded DB (work on local only) except for Redis and Mongo. At first, my first goal was to be a competitor of embedded DB like sqlite on Flutter. But Mongo storage can be something more. Cuz it supports remote and persistence, if some users just run free Mongo-atalas cloud service or run Mongo DB on some server, they can develop app service only with Flutter without backend but still using ANSI SQL. I think it can be competitive with firebase or supabase (There would be other issues like... we can't use convenient authentication though).

[fzyzcjy]

I see, now I understand the differences (it does seem to be quite different from things like SQLite etc), and that's pretty interesting!

[fzyzcjy]

(Reopen since the task of discussing translating the whole package is not finished thus to make it appear on my tracker)

[fzyzcjy]

Briefly checked the lifetime and other errors issue. I suggest to firstly /// flutter_rust_bridge:ignore on most of the mods and functions, pass the compilation, and add back things batch by batch. Then, it is easy to spot which function causes the problem. In addition, feel free to PR if you find some bug! Alternatively just create an issue and I will check it later.

[devgony]

After i upgraded frb to 2.0.0, my flutter_rust_bridge_codegen generate got error:

src/library/codegen/ir/hir/tree/module.rs:41:41: index out of bounds: the len is 0 but the index is 0

When i tried with master branch, it got

[2024-07-06T06:51:37.412Z ERROR frb_codegen/src/library/utils/logs.rs:55] panicked at frb_codegen/src/library/codegen/ir/mir/ty/structure.rs:45:17:
no entry found for key=MirStructIdent(NamespacedName { namespace: Namespace { joined_path: "gluesql_core::ast_builder" }, name: "PrimaryKeyNode" })
thread 'main' panicked at frb_codegen/src/library/codegen/ir/mir/ty/structure.rs:45:17:

All i did was just adding rust_input: crate::api,gluesql-core to frb_example/dart_minimal/flutter_rust_bridge.yaml and cargo add gluesql-core on frb_example/dart_minimal/rust/

It did not happened with previous 2.0.0.-dev42 version. Can you guess why?

[fzyzcjy]

Firstly, try to run frb on master branch which contain some fixes that may be related (I will publish 2.1.0 soon). If still error, feel free to create an issue!