No text detected with a physical Android device

[cswkim]

Using the latest version 0.0.5 and a physical Android phone on Android 11 API 30 I can't seem to get the example to detect text. This util method returns a VisionText instance that should have a list of blocks/lines/elements but it always returns an empty [] for it's blocks field. I have no issues using iOS. I'm not sure how to debug further, does anyone else have this issue?

flutter doctor -v

[✓] Flutter (Channel stable, 2.2.1, on Mac OS X 10.15.7 19H1030 darwin-x64, locale en-US)
    • Flutter version 2.2.1 at /Users/cswkim/Dev/flutter
    • Framework revision 02c026b03c (4 days ago), 2021-05-27 12:24:44 -0700
    • Engine revision 0fdb562ac8
    • Dart version 2.13.1

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/cswkim/Library/Android/sdk
    • Platform android-29, build-tools 29.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.56.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.22.0

[Soumajit2004]

i have same problem

[brianmtully]

are you detecting from camera or from an image?

[brianmtully]

Running the example application on Android 11, I get text results for both camera and picture scanning.

[cswkim]

are you detecting from camera or from an image?

I'm using the official camera plugin ^0.8.1 and the stream listener startImageStream((CameraImage image) {}). There is no additional set-up/config required for Android right? Setting any sort of gradle variable, etc.

[brianmtully]

Have you tried the example application in the repository?

[brianmtully]

Also, what ResolutionPreset are you using for the camera?

[cswkim]

I generally use ResolutionPreset.medium. Just as a test I tried it out on the Android Emulator for MacOS using a Pixel 3 device with API 29 and it seems to work fine. I'll fiddle around with it on another physical Android phone and report back.

[cswkim]

Ah strange. So on the physical Android device (Samsung Galaxy S10+), ResolutionPreset.medium produces no results for text detection, but the other values work fine: (low, high, veryHigh). Default behavior for the camera plugin is to select the next available lower resolution if the chosen is not available. I'm guessing that is not what is going on here.

[cswkim]

I decided to come back to this to try and figure out the issue and I might have stumbled upon the problem. The root of the issue is way over my head and involves knowledge of image processing, but I did find this article in case anyone is interested in the topic: https://www.programmersought.com/article/94353620149/

A more relevant link can be found in a prior open discussion on the now deprecated firebase_ml_vision library: https://github.com/FirebaseExtended/flutterfire/discussions/5575#discussioncomment-559068

This issue only seems to effect Android devices (and which ones specifically is a mixed bag). Here are some logging outputs for this util method:

Samsung Galaxy

ResolutionPreset.low image.planes[0].bytesPerRow.toDouble(): 320.0 image.width.toDouble(): 320.0

ResolutionPreset.medium image.planes[0].bytesPerRow.toDouble(): 768.0 image.width.toDouble(): 720.0

ResolutionPreset.high image.planes[0].bytesPerRow.toDouble(): 1280.0 image.width.toDouble(): 1280.0

As you can see, for medium, the bytesPerRow has an extra 48.0. If you set the width here to: size: Size(image.planes[0].bytesPerRow.toDouble(), image.height.toDouble()), text detection now works with ResolutionPreset.medium. You DO NOT want to do this for iOS, this is an Android-specific hack.

I've only tried this out on 2 separate Samsung Galaxy physical devices and am not sure what other Android phones need to address the issue. More importantly, this problem seems to be a deeper Android-related image processing quirk and not a bug of this package. I'm not sure how I will be attempting to apply this hack, especially since I'd rather avoid having to place some ugly Platform check conditional.

Anyways, I should have put a TL;DR disclaimer. One last note, I tried to set the imageFormatGroup parameter for the camera controller to ImageFormatGroup.jpeg to see if I could avoid this issue as Android defaults to ImageFormatGroup.yuv420 but it made things worse, text detection did not work at all, for any resolution preset.