[Bug] Possible Race Condition when using StreamSink setting methods

[qdot]

Describe the bug

Example repo: https://github.com/qdot/flutter-rust-bridge-streamsink-race

Specific files of interest:

• Main Dart File: https://github.com/qdot/flutter-rust-bridge-streamsink-race/blob/master/lib/main.dart
• Rust API File: https://github.com/qdot/flutter-rust-bridge-streamsink-race/blob/master/rust/src/api.rs

Whenever a stream creation method is defined in rust, the matching dart method returns a Stream<T>, which isn't wrapped in a future. This means that, if the rust method takes some time to run, we can get our stream back before it's finished being stored (for instance, in a lazy static OnceCell) and try to run operations on it which will result in errors, since no stream is actually stored yet. This method does not seem to run synchronously either.

The example repo shows the simplest repro I could make for this. The main.dart file loads the dynamic library, makes the call to create the stream, then goes on to call another FFI method that should add an element to the stream. However, without a sleep statement, the second FFI call will fail because no stream is available in the OnceCell yet; the original FFI call is still in the middle of setting the value.

Output with no sleep:

Changing current working directory to: C:\Users\qdot\code\flutter_rust_test
Launching lib\main.dart on Windows in debug mode...
Building Windows application...
flutter: Dylib Opened
flutter: Library Sink Set
flutter: Sink Event Handler set
In try sink!
NO STREAM AVAILABLE ON CALL
STREAM AVAILABLE ON CREATION
flutter: CLI running
flutter: Testing 1 2 3 on creation!

Output with sleep:

Changing current working directory to: C:\Users\qdot\code\flutter_rust_test
Launching lib\main.dart on Windows in debug mode...
Building Windows application...
flutter: Dylib Opened
flutter: Library Sink Set
STREAM AVAILABLE ON CREATION
flutter: Sink Event Handler set
In try sink!
STREAM AVAILABLE ON CALL
flutter: Testing 1 2 3!
flutter: CLI running

Codegen logs with RUST_LOG=debug environment variable

C:\Users\qdot\code\flutter_rust_test\rust [master]> flutter_rust_bridge_codegen --rust-input ./src/api.rs --dart-output ../lib/bridge_generated.dart
[2022-07-25T02:25:21Z DEBUG flutter_rust_bridge_codegen] configs=[Opts { rust_input_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\./src/api.rs", dart_output_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\../lib/bridge_generated.dart", dart_decl_output_path: None, c_output_path: ["C:\\Users\\qdot\\AppData\\Local\\Temp\\.tmpS9e22B.h"], rust_crate_dir: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust", rust_output_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\./src\\bridge_generated.rs", class_name: "Rust", dart_format_line_length: 80, skip_add_mod_to_lib: false, llvm_path: ["/opt/homebrew/opt/llvm", "/usr/local/opt/llvm", "/usr/lib/llvm-9", "/usr/lib/llvm-10", "/usr/lib/llvm-11", "/usr/lib/llvm-12", "/usr/lib/llvm-13", "/usr/lib/llvm-14", "/usr/lib/", "/usr/lib64/", "C:/Program Files/llvm", "C:/Program Files/LLVM", "C:/msys64/mingw64"], llvm_compiler_opts: "", manifest_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\Cargo.toml", dart_root: Some("C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\.."), build_runner: true, block_index: BlockIndex(0) }]
[2022-07-25T02:25:22Z DEBUG lib_flutter_rust_bridge_codegen::source_graph] Trying to parse "\\\\?\\C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\src\\bridge_generated.rs"
[2022-07-25T02:25:22Z DEBUG lib_flutter_rust_bridge_codegen::source_graph] Trying to parse "\\\\?\\C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\src\\api.rs"
[2022-07-25T02:25:22Z DEBUG lib_flutter_rust_bridge_codegen::parser] parse_function function name: Ident(try_sink)
[2022-07-25T02:25:22Z DEBUG lib_flutter_rust_bridge_codegen::parser] parse_function function name: Ident(set_sink)
[2022-07-25T02:25:22Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute command: bin=powershell args=["-noprofile", "-c", "dart pub global list"] current_dir=None cmd="powershell" "-noprofile" "-c" "dart pub global list"
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::commands] command="powershell" "-noprofile" "-c" "dart pub global list" stdout=ffigen 6.0.1
     stderr=
[2022-07-25T02:25:23Z INFO  lib_flutter_rust_bridge_codegen] Picked config: Opts { rust_input_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\./src/api.rs", dart_output_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\../lib/bridge_generated.dart", dart_decl_output_path: None, c_output_path: ["C:\\Users\\qdot\\AppData\\Local\\Temp\\.tmpS9e22B.h"], rust_crate_dir: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust", rust_output_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\./src\\bridge_generated.rs", class_name: "Rust", dart_format_line_length: 80, skip_add_mod_to_lib: false, llvm_path: ["/opt/homebrew/opt/llvm", "/usr/local/opt/llvm", "/usr/lib/llvm-9", "/usr/lib/llvm-10", "/usr/lib/llvm-11", "/usr/lib/llvm-12", "/usr/lib/llvm-13", "/usr/lib/llvm-14", "/usr/lib/", "/usr/lib64/", "C:/Program Files/llvm", "C:/Program Files/LLVM", "C:/msys64/mingw64"], llvm_compiler_opts: "", manifest_path: "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\Cargo.toml", dart_root: Some("C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\.."), build_runner: true, block_index: BlockIndex(0) }
[2022-07-25T02:25:23Z INFO  lib_flutter_rust_bridge_codegen] Phase: Parse source code to AST, then to IR
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::source_graph] Trying to parse "\\\\?\\C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\src\\bridge_generated.rs"
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::source_graph] Trying to parse "\\\\?\\C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\src\\api.rs"
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::parser] parse_function function name: Ident(try_sink)
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::parser] parse_function function name: Ident(set_sink)
[2022-07-25T02:25:23Z INFO  lib_flutter_rust_bridge_codegen] Phase: Transform IR
[2022-07-25T02:25:23Z INFO  lib_flutter_rust_bridge_codegen] Phase: Generate Rust code
[2022-07-25T02:25:23Z INFO  lib_flutter_rust_bridge_codegen] Phase: Generate Dart code
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::generator::dart] distinct_input_types=[]
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::generator::dart] distinct_output_types=[Delegate(String), Primitive(U8), PrimitiveList(IrTypePrimitiveList { primitive: U8 }), Primitive(Unit)]
[2022-07-25T02:25:23Z INFO  lib_flutter_rust_bridge_codegen] Phase: Other things
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute format_rust path=C:\Users\qdot\code\flutter_rust_test\rust\./src\bridge_generated.rs
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute command: bin=rustfmt args=["C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\./src\\bridge_generated.rs"] current_dir=None cmd="rustfmt" "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\./src\\bridge_generated.rs"
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::commands] command="rustfmt" "C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\./src\\bridge_generated.rs" stdout= stderr=
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute cbindgen rust_crate_dir=C:\Users\qdot\code\flutter_rust_test\rust c_output_path=C:\Users\qdot\AppData\Local\Temp\.tmpbH1sxG.h
[2022-07-25T02:25:23Z DEBUG lib_flutter_rust_bridge_codegen::commands] cbindgen config: Config { header: None, includes: [], sys_includes: ["stdbool.h", "stdint.h", "stdlib.h"], after_includes: None, trailer: None, include_guard: None, pragma_once: false, no_includes: true, autogen_warning: None, include_version: false, namespace: None, namespaces: None, using_namespaces: None, braces: SameLine, line_length: 100, tab_width: 2, line_endings: LF, language: C, cpp_compat: false, style: Both, sort_by: None, usize_is_size_t: false, parse: ParseConfig { parse_deps: false, include: None, exclude: [], expand: ParseExpandConfig { crates: [], all_features: false, default_features: true, features: None, profile: Debug }, clean: false, extra_bindings: [] }, export: ExportConfig { include: [], exclude: [], rename: {}, pre_body: {}, body: {}, prefix: None, item_types: [], renaming_overrides_prefixing: false, mangle: MangleConfig { rename_types: None, remove_underscores: false } }, macro_expansion: MacroExpansionConfig { bitflags: false }, layout: LayoutConfig { packed: None, aligned_n: None }, function: FunctionConfig { prefix: None, postfix: None, must_use: None, args: Auto, rename_args: None, swift_name_macro: None, sort_by: None, no_return: None }, structure: StructConfig { rename_fields: None, derive_constructor: false, derive_eq: false, derive_neq: false, derive_lt: false, derive_lte: false, derive_gt: false, derive_gte: false, derive_ostream: false, associated_constants_in_body: false, must_use: None }, enumeration: EnumConfig { rename_variants: None, rename_variant_name_fields: SnakeCase, add_sentinel: false, prefix_with_name: false, derive_helper_methods: false, derive_const_casts: false, derive_mut_casts: false, cast_assert_name: None, must_use: None, derive_tagged_enum_destructor: false, derive_tagged_enum_copy_constructor: false, derive_tagged_enum_copy_assignment: false, derive_ostream: false, enum_class: true, private_default_tagged_enum_constructor: false }, constant: ConstantConfig { allow_static_const: true, allow_constexpr: true, sort_by: None }, defines: {}, documentation: true, documentation_style: Auto, documentation_length: Full, pointer: PtrConfig { non_null_attribute: None }, only_target_dependencies: false, cython: CythonConfig { header: None, cimports: {} } }
[2022-07-25T02:25:24Z DEBUG cbindgen::bindgen::parser] Parsing crate rust
[2022-07-25T02:25:24Z INFO  cbindgen::bindgen::parser] Take rust::wire_try_sink.
[2022-07-25T02:25:24Z INFO  cbindgen::bindgen::parser] Take rust::wire_set_sink.
[2022-07-25T02:25:24Z INFO  cbindgen::bindgen::parser] Take rust::free_WireSyncReturnStruct.
[2022-07-25T02:25:24Z INFO  cbindgen::bindgen::parser] Take rust::DartPort.
[2022-07-25T02:25:24Z INFO  cbindgen::bindgen::parser] Take rust::DartPostCObjectFnType.
[2022-07-25T02:25:24Z INFO  cbindgen::bindgen::parser] Take rust::store_dart_post_cobject.
[2022-07-25T02:25:24Z INFO  cbindgen::bindgen::parser] Take rust::WireSyncReturnStruct.
[2022-07-25T02:25:24Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute ffigen c_path=C:\Users\qdot\AppData\Local\Temp\.tmpbH1sxG.h dart_path=C:\Users\qdot\AppData\Local\Temp\.tmpeJokb3 llvm_path=["/opt/homebrew/opt/llvm", "/usr/local/opt/llvm", "/usr/lib/llvm-9", "/usr/lib/llvm-10", "/usr/lib/llvm-11", "/usr/lib/llvm-12", "/usr/lib/llvm-13", "/usr/lib/llvm-14", "/usr/lib/", "/usr/lib64/", "C:/Program Files/llvm", "C:/Program Files/LLVM", "C:/msys64/mingw64"]
[2022-07-25T02:25:24Z DEBUG lib_flutter_rust_bridge_codegen::commands] ffigen config:
            output: 'C:\Users\qdot\AppData\Local\Temp\.tmpeJokb3'
            name: 'RustWire'
            description: 'generated by flutter_rust_bridge'
            headers:
              entry-points:
                - 'C:\Users\qdot\AppData\Local\Temp\.tmpbH1sxG.h'
              include-directives:
                - 'C:\Users\qdot\AppData\Local\Temp\.tmpbH1sxG.h'
            comments: false
            preamble: |
              // ignore_for_file: camel_case_types, non_constant_identifier_names, avoid_positional_boolean_parameters, annotate_overrides, constant_identifier_names

            llvm-path:
               - '/opt/homebrew/opt/llvm'
               - '/usr/local/opt/llvm'
               - '/usr/lib/llvm-9'
               - '/usr/lib/llvm-10'
               - '/usr/lib/llvm-11'
               - '/usr/lib/llvm-12'
               - '/usr/lib/llvm-13'
               - '/usr/lib/llvm-14'
               - '/usr/lib/'
               - '/usr/lib64/'
               - 'C:/Program Files/llvm'
               - 'C:/Program Files/LLVM'
               - 'C:/msys64/mingw64'

[2022-07-25T02:25:24Z DEBUG lib_flutter_rust_bridge_codegen::commands] ffigen config_file: NamedTempFile("C:\\Users\\qdot\\AppData\\Local\\Temp\\.tmpiSUY6h")
[2022-07-25T02:25:24Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute command: bin=powershell args=["-noprofile", "-c", "dart pub global run ffigen --config \"C:\\Users\\qdot\\AppData\\Local\\Temp\\.tmpiSUY6h\""] current_dir=None cmd="powershell" "-noprofile" "-c" "dart pub global run ffigen --config \"C:\\Users\\qdot\\AppData\\Local\\Temp\\.tmpiSUY6h\""
[2022-07-25T02:25:26Z DEBUG lib_flutter_rust_bridge_codegen::commands] command="powershell" "-noprofile" "-c" "dart pub global run ffigen --config \"C:\\Users\\qdot\\AppData\\Local\\Temp\\.tmpiSUY6h\"" stdout=Running in Directory: 'C:\Users\qdot\code\flutter_rust_test\rust'
    Input Headers: [C:\Users\qdot\AppData\Local\Temp\.tmpbH1sxG.h]
    Finished, Bindings generated in C:\Users\qdot\AppData\Local\Temp\.tmpeJokb3
     stderr=
[2022-07-25T02:25:26Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute format_dart path=C:\Users\qdot\code\flutter_rust_test\rust\../lib/bridge_generated.dart line_length=80
[2022-07-25T02:25:26Z DEBUG lib_flutter_rust_bridge_codegen::commands] execute command: bin=powershell args=["-noprofile", "-c", "dart format C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\../lib/bridge_generated.dart --line-length 80"] current_dir=None cmd="powershell" "-noprofile" "-c" "dart format C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\../lib/bridge_generated.dart --line-length 80"
[2022-07-25T02:25:27Z DEBUG lib_flutter_rust_bridge_codegen::commands] command="powershell" "-noprofile" "-c" "dart format C:\\Users\\qdot\\code\\flutter_rust_test\\rust\\../lib/bridge_generated.dart --line-length 80" stdout=Formatted C:\Users\qdot\code\flutter_rust_test\rust\../lib/bridge_generated.dart
    Formatted 1 file (1 changed) in 0.20 seconds.
     stderr=
[2022-07-25T02:25:27Z INFO  lib_flutter_rust_bridge_codegen] Success!
[2022-07-25T02:25:27Z INFO  flutter_rust_bridge_codegen] Now go and use it :)

To Reproduce

No response

Expected behavior

No response

Generated binding code

No response

OS

All (Tested on Windows and Android)

Version of flutter_rust_bridge_codegen

1.38.1

Flutter info

[✓] Flutter (Channel stable, 3.0.5, on Microsoft Windows [Version 10.0.19044.1826], locale en-US)
    • Flutter version 3.0.5 at c:\Users\qdot\Programs\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f1875d570e (11 days ago), 2022-07-13 11:24:16 -0700
    • Engine revision e85ea0e79c
    • Dart version 2.17.6
    • DevTools version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at C:\Users\qdot\AppData\Local\Android\sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.12+7-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

[✓] Visual Studio - develop for Windows (Visual Studio Community 2019 16.10.4)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.10.31515.178
    • Windows 10 SDK version 10.0.22000.0

[✓] Android Studio (version 2021.2)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+7-b1504.28-7817840)

[✓] VS Code (version 1.69.2)
    • VS Code at C:\Users\qdot\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.44.0

[✓] Connected device (4 available)
    • Pixel 3a XL (mobile) • *********  • android-arm64  • Android 12 (API 32)
    • Windows (desktop)    • windows    • windows-x64    • Microsoft Windows [Version 10.0.19044.1826]
    • Chrome (web)         • chrome     • web-javascript • Google Chrome 103.0.5060.134
    • Edge (web)           • edge       • web-javascript • Microsoft Edge 97.0.1072.69

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

Version of clang++

14.0.6

Version of ffigen

No response

Additional context

No response

[welcome[bot]]

Hi! Thanks for opening your first issue here! :smile:

[fzyzcjy]

Nice reproduction and clear description :) Let me examine it

[fzyzcjy]

I see the problem. So I guess maybe the simplest solution is:

1. Let rust code add some data (probably a sentinel) into streamsink as long as it feels it is ready.
2. On dart side, do not consider it is safe to call cli.trySink, until dart receive that sentinel value

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new issue.