iOS Imagestream: Manage list of planes

[stefanodecillis]

ImageStreamBuilder iOS

Hi,

I'm trying to implement a QR code reader with CameraAwesome. On Android, it works just fine. On iOS it doesn't work.

In details:

In imageStreamBuilder, I get the streamSubscription and I start scanner calling the function "onData". In this case I convert the data given by the stream in a File and then I use BarcodeDetector (Google ML kit).

I get the following error:

021-01-07 19:00:58.055465+0100 Runner[2147:9132611] *** Terminating app due to uncaught exception 'FIRInvalidImage', reason: 'Input image must not be nil.' *** First throw call stack: (0x18a51d86c 0x19f48cc50 0x18a4164a4 0x1012e9934 0x101503624 0x1015033a0 0x101502d84 0x101502994 0x103da94e8 0x103568178 0x1038429ec 0x10357170c 0x103573e38 0x18a499fa0 0x18a499ba0 0x18a498ffc 0x18a492ee4 0x18a49221c 0x1a1f96784 0x18ced0fe0 0x18ced6854 0x100d646e0 0x18a1526b0) *** Terminating app due to uncaught exception 'FIRInvalidImage', reason: 'Input image must not be nil.' terminating with uncaught exception of type NSException libc++abi.dylib: terminating with uncaught exception of type NSException

Since it works smoothly in Android, I think there is a problem with iOS. The problem can be reproduced with the following code:

CameraAwesome(
                    sensor: _sensor,
                    captureMode: _captureMode,
                    photoSize: _photoSize,
                    selectDefaultSize: (List<Size> availableSizes) =>
                        Size(1920, 1080),
                    imagesStreamBuilder: (Stream<Uint8List> imageStream) {
                      // ignore: cancel_subscriptions
                      setState(() {
                        this.cameraMainStream = imageStream;
                      });

                      StreamSubscription<Uint8List> streamSubscription =
                          imageStream.listen((Uint8List imageData) {});

                      setState(() {
                        this.cameraStream = streamSubscription;
                      });

                      streamSubscription.onData((data) async {
                        if (data == null) return;
                        if (skipFrames) { //used to low the overhead. Not necessary to reproduce the problem
                          return;
                        }

                        skipFrame(true);

                        final path = join(
                          (await getTemporaryDirectory()).path,
                          '${DateTime.now()}.png',
                        );
                        debugPrint('3');
                        File file = File(path);
                        file.writeAsBytesSync(data);
                        await file.writeAsBytes(data);
                        //File file = File.fromRawPath(data);
                        if (file == null) {
                          debugPrint('-------> NULL');
                        }
                        if (await file.exists()) {
                          debugPrint('exists');
                        }

                        //File file = File.fromRawPath(data);
                        try {
                          File fileCopy = File(path);
                          FirebaseVisionImage visionImage =
                              FirebaseVisionImage.fromFile(fileCopy);
                          if (visionImage == null) {
                            debugPrint('-------> NULL');
                          }

                          String result =
                              await QrProvider.detectQR(visionImage); //this calls BarcodeDetector.detectInImage(image);
                         
                        } catch (e) {
                          debugPrint(e);
                        }

                        skipFrame(false);
                      });
                    },
                    onCameraStarted: () {
                      skipFrame(false);
                    },
                  )

I don't think it is the way I'm handling the data but probably you know it better. Can you help me with this problem? Thank you in advance!

[g-apparence]

Our iOS master will get a look 👍

[stefanodecillis]

Hi @g-apparence there is any news regarding the problem? I think this problem is related to the iOS data that the plugin is retrieving but I could be wrong

[g-apparence]

Hello, Sorry was really busy on a project. I'm asking for a fix this week!

[ghost]

Hi @stefanodecillis You need to convert image by using planes (this is for iOS only). Currently, it's not fully integrated because we need to find how to create a second listener with specific data without touching the Android one...

To be more clear, Android image listener only return Uint8List data but for iOS we need to return Uint8List and a list of planes (needed to convert data into an image).

Our final goal is to handle the image conversion directly inside the library in order to not use plane list.

We are investigating how to do this.

Thanks,

[stefanodecillis]

Hi @dim-apparence , thank you! I was surfing on the web and I couldn't find any example related to this. The closest example was using the CameraImage object. Can you provide me with an example to do it? It can be helpful to everyone that it's using the image streaming on iOS! Thank you again for your support!

[ghost]

Hi @stefanodecillis no problem :) you can see how the official camera plugin works in this tutorial.

The official camera plugin return Uint8List and a list of planes this is why it's working.

Currently in Camerawesome, planes are missing :/ I think we need to add them to follow the same tutorial.

You're welcome 👍

[stefanodecillis]

Oh okay, that why I couldn't find something related to this. I also looked all over your plugin and at the end, you are working with a specific platform plugin (I imagine that is always yours). If I may, can I ask you if it's something that is coming soon or it is still in the wishlist? The point is that I really like your plugin since it's way more stable than the original one 😭

[ghost]

@stefanodecillis Many thanks for your encouragements! 🥇

We are currently busy on other projects but this issue will be fix asap 👍

Our vision is to create a more "complex" plugin to hide all image data processing directly into the camerawesome plugin instead of the official one (see how long this tutorial is just to retrieve images from stream...). But this is difficult and time consuming...

[g-apparence]

Hi Stephano. You are right we completely rewrite the platform plugins to be able to create what we want. Also this is a great way to learn new things, so we took pleasure doing it :). We are glad you love our plugin. We didn't have any plan on this as we work on it every time we can (between two projects or free time...).
I Will put this as a max priority with the multi-camera feature on Android. 👍

[stefanodecillis]

Thank you guys for your support!

[istornz]

Hi @stefanodecillis, after a long long time, this feature was added 👍

Feel free to check it out 👍