Error when use plugin inside flutter isolate

[mpcreza]

Description I get an error when using the plugin inside an isolate. in my case, i use flutter_background_service plugin.

Expected behavior usable outside the main thread

Current behavior When call FFmpegKit.executeAsync, i get this error: Unhandled Exception: MissingPluginException(No implementation found for method getLogLevel on channel flutter.arthenica.com/ffmpeg_kit)

To Reproduce i created a simaple example

Screenshots

Logs

E/flutter ( 6408): [ERROR:flutter/lib/ui/ui_dart_state.cc(209)] Unhandled Exception: MissingPluginException(No implementation found for method getLogLevel on channel flutter.arthenica.com/ffmpeg_kit)
E/flutter ( 6408): #0      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:175:7)
E/flutter ( 6408): <asynchronous suspension>
E/flutter ( 6408): #1      FFmpegKitInitializer._initialize (package:ffmpeg_kit_flutter_audio/src/ffmpeg_kit_flutter_initializer.dart:313:22)
E/flutter ( 6408): <asynchronous suspension>
E/flutter ( 6408): #2      FFmpegKitInitializer.initialize (package:ffmpeg_kit_flutter_audio/src/ffmpeg_kit_flutter_initializer.dart:54:7)
E/flutter ( 6408): <asynchronous suspension>
E/flutter ( 6408): #3      FFmpegKitConfig.init (package:ffmpeg_kit_flutter_audio/ffmpeg_kit_config.dart:50:5)
E/flutter ( 6408): <asynchronous suspension>
E/flutter ( 6408): #4      AbstractSession.createFFmpegSession (package:ffmpeg_kit_flutter_audio/abstract_session.dart:69:7)
E/flutter ( 6408): <asynchronous suspension>
E/flutter ( 6408): #5      FFmpegSession.create (package:ffmpeg_kit_flutter_audio/ffmpeg_session.dart:40:21)
E/flutter ( 6408): <asynchronous suspension>
E/flutter ( 6408): #6      FFmpegKit.executeWithArguments (package:ffmpeg_kit_flutter_audio/ffmpeg_kit.dart:44:9)
E/flutter ( 6408): <asynchronous suspension>
E/flutter ( 6408): 

======== Exception caught by services library ======================================================
The following MissingPluginException was thrown while activating platform stream on channel flutter.arthenica.com/ffmpeg_kit_event:
MissingPluginException(No implementation found for method listen on channel flutter.arthenica.com/ffmpeg_kit_event)

When the exception was thrown, this was the stack: 
#0      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:175:7)
<asynchronous suspension>
#1      EventChannel.receiveBroadcastStream.<anonymous closure> (package:flutter/src/services/platform_channel.dart:516:9)
<asynchronous suspension>

Environment

• Platform: Android
• Architecture: x86
• Version: 4.5.1-LTS
• Android Studio version: 2021.1.1 Patch 2
• flutter doctor: Flutter (Channel stable, 2.10.3, on Fedora Linux 35 (Workstation Edition) 5.16.12-200.fc35.x86_64, locale en_US.UTF-8)

Other I tested with other plugins like device_info_plus and it worked without problems inside an isolate.

[mpcreza]

As far as I know anything that needs a UI should not be used in isolate. i don't know about IOS but in android The problem is probably because the plugin needs an activity to initialize

[mpcreza]

hey @tanersener, i forked ffmpeg-kit and made some small changes just for android and it worked inside isolate, But logs are not printed on the main thread, i don't know why. can you check it? maybe give you some cloue for fix this problem. foked ffmpeg-kit

[tanersener]

Thanks for creating this issue.

When I run example application given in the first post, I see the following logs about the plugin.

8416-8416/com.example.ffmpeg_kit_test D/ffmpeg-kit-flutter: FFmpegKitFlutterPlugin created com.arthenica.ffmpegkit.flutter.FFmpegKitFlutterPlugin@4cc9c7c.
8416-8416/com.example.ffmpeg_kit_test D/ffmpeg-kit-flutter: FFmpegKitFlutterPlugin com.arthenica.ffmpegkit.flutter.FFmpegKitFlutterPlugin@4cc9c7c attached to activity com.example.ffmpeg_kit_test.MainActivity@273376f.
8416-8416/com.example.ffmpeg_kit_test D/ffmpeg-kit-flutter: FFmpegKitFlutterPlugin com.arthenica.ffmpegkit.flutter.FFmpegKitFlutterPlugin@4cc9c7c initialised with context android.app.Application@62dd5a and activity com.example.ffmpeg_kit_test.MainActivity@273376f.
8416-8416/com.example.ffmpeg_kit_test D/ffmpeg-kit-flutter: FFmpegKitFlutterPlugin created com.arthenica.ffmpegkit.flutter.FFmpegKitFlutterPlugin@521c6a.

These logs simply say that two FFmpegKitFlutterPlugin instances are created. The first one (@4cc9c7c) is registered successfully. The second one (@521c6a), created by the isolate is not registered properly. That's why you have that error.

Even though you have a different stack trace than the original stack trace in #263, the root cause is the same. You are using an isolate. Isolates cannot call platform channels. This is the original ticket under the flutter project about this feature, flutter/flutter/issues/13937. We don't have a workaround for this. So, I suggest waiting an update from flutter devs.

I had a look at your second app as well. The problem is the same. Logs are forwarded by platform channels in ffmpeg-kit-flutter. When platform channels don't work. Logs also don't work.

[maks]

Given that FFmpeg is a native library, it would be both more efficient and solve this issue of usage within non-main Isolate, by calling FFmpeg via Dart FFI directly instead of going via a platform channel->Java->JNI->FFMpeg on Android.

My humble suggestion, looking at https://github.com/tanersener/ffmpeg-kit/blob/main/flutter/flutter/lib/src/ffmpeg_kit_flutter_initializer.dart and seeing the use of callbacks, you would probably actually want to always use an Isolate to do the FFI calls to prevent the main Isolate from blocking on sync calls into the FFmpegkit wrapper C code.

The JNI wrapper code appears to be in https://github.com/tanersener/ffmpeg-kit/blob/main/android/ffmpeg-kit-android-lib/src/main/cpp/ffmpegkit.h but of course that header file is just the awkward definitions required by JNI.

Looking at the very well documented C file for that header though, it seems that all the JNI function definitions are just wrappers around fairly nice C code: https://github.com/tanersener/ffmpeg-kit/blob/main/android/ffmpeg-kit-android-lib/src/main/cpp/ffmpegkit.c some of the functions are a bit involved while others are just JNI wrapping for returning a single variable. Given that perhaps it would not be too difficult to just make a Dart FFI version of that C file and a nice header to match, mainly just removing all the JNI required boilerplate.

Then essentially you would use Darts ffi-gen package on that header file to create the Dart FFI binding for it and then directly call into it from the existing Dart code avoiding altogether the need of going via the platform channel which then goes via the JNI code in https://github.com/tanersener/ffmpeg-kit/tree/main/android/ffmpeg-kit-android-lib/src/main/java/com/arthenica/ffmpegkit

I appreciate that the above is a fair bit of work, but given you have already built the Flutter platform channel interface and the JNI one, you would actually remove alot of the complexity in the current Flutter implementation for this very nice project.

There is also prior art that you could refer to for doing this, for example in this plugin: https://github.com/InvisibleWrench/flutter_midi_command_linux/blob/master/lib/flutter_midi_command_linux.dart there is an implementation of using the ALSA Midi interface via Dart FFI, running within a separate Isolate so as not to block the main Flutter Isolate.

[tanersener]

Thanks for your suggestion, and for the analysis regarding the current implementation.

We have considered using Dart FFI a few times before. As far as I remember, it was first suggested to improve the performance by @aytunch. I wasn't expecting to have an improvement in encoding speed, so we didn't spend time on it.

I agree that it can address the problems we have about using the library with isolates.

But, there are a few challenges there. Because of them, Dart FFI is not high in the project task list. But we are open to PRs if someone wants to implement it.

The main issue is;

• Flutter plugin of ffmpeg-kit is a wrapper of the native ffmpeg-kit libraries (Android/iOS/macOS)
• And, contrary to what most devs believe, ffmpeg-kit native libraries (Android/iOS/macOS) are not just wrappers of ffmpeg. There is logic inside
• Not using the JNI part of ffmpeg-kit Android library means, implementing the same mechanisms in Flutter plugin with Dart FFI
• This is the problem. That won't be easy

There are also some risks about SAF (Storage Access Framework) API methods we have for Android. We are calling Android's Java API inside them. They don't have native (Android NDK) equivalents. So, I'm not sure if we can implement them if we switch to Dart FFI.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Silfalion]

Hello, just checking in on this issue, since Flutter supports method channels in isolates now, this should work, right?

I'm facing an issue while using an isolate with the package:

Bad state: The BackgroundIsolateBinaryMessenger.instance value is invalid until BackgroundIsolateBinaryMessenger.ensureInitialized is executed.

Would there be a recommended workaround for it, please?

[ShahoodulHassan]

@tanersener

You are using an isolate. Isolates cannot call platform channels.

Since Flutter 3.7 onwards now supports platform channels in isolates, can this issue be resolved?

If not, what other workaround there is to use this plugin in a background thread for, let's say, compressing videos?

[tanersener]

@ShahoodulHassan That's correct. We will update the Flutter plugin. We just don't have an ETA yet.

[svillacreses]

@ShahoodulHassan That's correct. We will update the Flutter plugin. We just don't have an ETA yet.

Any updates on this? :(

[kaciula]

This issue should remain open until this is actually implemented in the plugin.

One of my apps is doing a lot of video processing and the app is laggy on account I can't run the various ffmpeg commands in a background isolate.

If this issue isn't being addressed, one alternative would be to start a new Flutter engine and run the ffmpeg commands on the root isolate there. It seems overkill and will be definitely slower than just simply starting a background isolate.

[Henriquek47]

Any updated?

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[ShahoodulHassan]

It needs to be kept open.

[Henriquek47]

In the new Flutter 3.29 version, threads have been unified, now running the UI and native code on the same thread. This was one of the obstacles to running FFmpeg in an isolate, correct?

[Silfalion]

I missed that @Henriquek47, could you provide a link, pls? I'm curious about the breaking changes it will create.

[Henriquek47]

@Silfalion https://medium.com/flutter/whats-new-in-flutter-3-29-f90c380c2317#:~:text=dart%20source%20code.-,Dart%20threading%20changes%20on%20Android/iOS,-Previously%2C%20Flutter%20executed

[vuminhhieucareer172]

Anything new?? :(