flutter_google_ml_vision
flutter_google_ml_vision copied to clipboard
No text detected with a physical Android device
Using the latest version 0.0.5
and a physical Android phone on Android 11 API 30
I can't seem to get the example to detect text. This util method returns a VisionText
instance that should have a list of blocks/lines/elements
but it always returns an empty []
for it's blocks
field. I have no issues using iOS
. I'm not sure how to debug further, does anyone else have this issue?
flutter doctor -v
[✓] Flutter (Channel stable, 2.2.1, on Mac OS X 10.15.7 19H1030 darwin-x64, locale en-US)
• Flutter version 2.2.1 at /Users/cswkim/Dev/flutter
• Framework revision 02c026b03c (4 days ago), 2021-05-27 12:24:44 -0700
• Engine revision 0fdb562ac8
• Dart version 2.13.1
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
• Android SDK at /Users/cswkim/Library/Android/sdk
• Platform android-29, build-tools 29.0.2
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 12.4, Build version 12D4e
• CocoaPods version 1.10.1
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 4.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
[✓] VS Code (version 1.56.2)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.22.0
i have same problem
are you detecting from camera or from an image?
Running the example application on Android 11, I get text results for both camera and picture scanning.
are you detecting from camera or from an image?
I'm using the official camera
plugin ^0.8.1
and the stream listener startImageStream((CameraImage image) {})
. There is no additional set-up/config required for Android right? Setting any sort of gradle variable, etc.
Have you tried the example application in the repository?
Also, what ResolutionPreset are you using for the camera?
I generally use ResolutionPreset.medium
. Just as a test I tried it out on the Android Emulator for MacOS using a Pixel 3 device with API 29
and it seems to work fine. I'll fiddle around with it on another physical Android phone and report back.
Ah strange. So on the physical Android device (Samsung Galaxy S10+), ResolutionPreset.medium
produces no results for text detection, but the other values work fine: (low
, high
, veryHigh
). Default behavior for the camera
plugin is to select the next available lower resolution if the chosen is not available. I'm guessing that is not what is going on here.
I decided to come back to this to try and figure out the issue and I might have stumbled upon the problem. The root of the issue is way over my head and involves knowledge of image processing, but I did find this article in case anyone is interested in the topic: https://www.programmersought.com/article/94353620149/
A more relevant link can be found in a prior open discussion on the now deprecated firebase_ml_vision
library: https://github.com/FirebaseExtended/flutterfire/discussions/5575#discussioncomment-559068
This issue only seems to effect Android devices (and which ones specifically is a mixed bag). Here are some logging outputs for this util method:
Samsung Galaxy
ResolutionPreset.low
image.planes[0].bytesPerRow.toDouble()
: 320.0
image.width.toDouble()
: 320.0
ResolutionPreset.medium
image.planes[0].bytesPerRow.toDouble()
: 768.0
image.width.toDouble()
: 720.0
ResolutionPreset.high
image.planes[0].bytesPerRow.toDouble()
: 1280.0
image.width.toDouble()
: 1280.0
As you can see, for medium
, the bytesPerRow
has an extra 48.0. If you set the width here to:
size: Size(image.planes[0].bytesPerRow.toDouble(), image.height.toDouble()),
text detection now works with ResolutionPreset.medium
. You DO NOT want to do this for iOS, this is an Android-specific hack.
I've only tried this out on 2 separate Samsung Galaxy physical devices and am not sure what other Android phones need to address the issue. More importantly, this problem seems to be a deeper Android-related image processing quirk and not a bug of this package. I'm not sure how I will be attempting to apply this hack, especially since I'd rather avoid having to place some ugly Platform check conditional.
Anyways, I should have put a TL;DR disclaimer. One last note, I tried to set the imageFormatGroup
parameter for the camera controller to ImageFormatGroup.jpeg
to see if I could avoid this issue as Android defaults to ImageFormatGroup.yuv420
but it made things worse, text detection did not work at all, for any resolution preset.