Problem loading Google's Teachable Machine models

[cmalbuquerque]

I would like to know if these known issues will be resolved in the meantime?

• App crashes when runnning GTM model on both android and iOS - To reduce your app's footprint, this package has disabled GTM feature by default
• App crashes when running GTM model on iOS emulator - please run your simulation on actual iOS device. Tflite for x86_64 architectures have limited support

[Caldarie]

Hi @cmalbuquerque

If I may confirm the first issue - is the problem the binary size? Or manually enabling GTM?

For the second issue, I believe the authors responsible for tensorflow iOS are investigating the problem, which can be found here. If you absolutely need to use an emulator, you can consider building the select ops framework yourself. Instructions can be provided here

[cmalbuquerque]

@Caldarie thanks for your tips! 😁

For the first issue, I was having problems loading the GTM model but already fixed it. Works on Android device but I did not yet tested on iOS!

To use an emulator, I have been having some problems configuring correctly but I will guide using the documentation link you suggested! Thanks again 🤩

[Caldarie]

@cmalbuquerque No problems at all. I’m glad it worked out in the end.

If it’s not too much trouble, would you mind sharing how you fixed he problem? It would really help others who also had a similar problem as you.

[cmalbuquerque]

Basically, I follow these steps first (for Android scenario):

1. Add permissions to Android Manifest

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

2. Add aaptOptions to app/build.gradle file

android {
   (...)
    aaptOptions {
        noCompress 'tflite'
    }

    lintOptions {
        disable 'InvalidPackage'
    }
(...)
}

3. Enable select-ops in app/build.gradle file

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation "org.tensorflow:tensorflow-lite-select-tf-ops:+"
}

After that, I loaded the model using the following definitions in the initState() :

    TfliteAudio.loadModel(
        model: 'assets/soundclassifier.tflite',
        label: 'assets/labels.txt',
        numThreads: 1,
        isAsset: true);

And to get the result I specified 'rawAudio' has inputType because of GTM model's input type, and in order to make a quick test I defined a recordingLength. After completed the recording length, the classification is printed on terminal.

void getResults() {
    result = TfliteAudio.startAudioRecognition(
      numOfInferences: 1,
      inputType: 'rawAudio',
      sampleRate: 44100,
      recordingLength: 44032,
      bufferSize: 22016,
    );
    result.listen((event) {
      print(event.toString());
    }).onDone(() => isRecording.value = false);
  }

I only tested with an Android device, but I will follow these steps to prepare iOS device.

[Caldarie]

@cmalbuquerque many thanks for the detailed explanation!

[Tanelo]

Hey guys I wanted to know if its is working on iOs now ??

[Caldarie]

@Tanelo iOS should work if you run it on a device and not emulator. Follow the instructions here to install

[Tanelo]

@Caldarie not working on Android for me ..

[Caldarie]

@Tanelo - can you provide details to reproduce this problem? It’s difficult to assist you if you don’t give more information

Also, have you tried running the example as well to check where you went wrong? You can git clone this project and run the example on your device. To do that, run the following commands below on terminal:

1. git clone https://github.com/Caldarie/flutter_tflite_audio.git
2. cd <example folder location>
3. flutter run

[Tanelo]

@Caldarie Thanks for your help, I've obviously cloned the repo. I've managed to run the app using the "decodwav" dataset but the GTM is not working, even on Android. There are like 155errrors on running the GMT dataset.. The end is of the debug session : ... #146 pc 00379b00 /apex/com.android.art/lib/libart.so (art::interpreter::EnterInterpreterFromEntryPoint(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*)+176) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #147 pc 0078b325 /apex/com.android.art/lib/libart.so (artQuickToInterpreterBridge+1061) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #148 pc 0014220d /apex/com.android.art/lib/libart.so (art_quick_to_interpreter_bridge+77) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #149 pc 00893656 /system/framework/x86/boot-framework.oat (com.android.internal.os.ZygoteInit.main+2102) (BuildId: 9a9778e61b43d349325d0bb85244bd9bc95ff387) #150 pc 0013baf2 /apex/com.android.art/lib/libart.so (art_quick_invoke_static_stub+418) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #151 pc 001d0392 /apex/com.android.art/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+258) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #152 pc 0062e653 /apex/com.android.art/lib/libart.so (art::JValue art::InvokeWithVarArgsart::ArtMethod*(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, art::ArtMethod*, char*)+579) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #153 pc 0062eb25 /apex/com.android.art/lib/libart.so (art::JValue art::InvokeWithVarArgs<_jmethodID*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, char*)+85) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #154 pc 004ce64f /apex/com.android.art/lib/libart.so (art::JNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, char*)+735) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #155 pc 003f8aae /apex/com.android.art/lib/libart.so (art::(anonymous namespace)::CheckJNI::CallMethodV(char const*, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, char*, art::Primitive::Type, art::InvokeType)+2846) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #156 pc 003e60d9 /apex/com.android.art/lib/libart.so (art::(anonymous namespace)::CheckJNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, char*)+73) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #157 pc 0008f90e /system/lib/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+62) (BuildId: 588f2cd5873ff4273bb25b25edb82606) #158 pc 00098c8e /system/lib/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vectorandroid::String8 const&, bool)+910) (BuildId: 588f2cd5873ff4273bb25b25edb82606) #159 pc 00003804 /system/bin/app_process32 (main+1588) (BuildId: c5eedbfb6130af84c3db8e121fb1202e) #160 pc 000522e3 /apex/com.android.runtime/lib/bionic/libc.so (__libc_init+115) (BuildId: 6e3a0180fa6637b68c0d181c343e6806) Lost connection to device.

The beginning is `Launching lib/main.dart on sdk gphone x86 arm in debug mode... lib/main.dart:1 ✓ Built build/app/outputs/flutter-apk/app-debug.apk. Connecting to VM Service at ws://127.0.0.1:56505/L9u8r_LBthI=/ws D/Tflite_audio( 4423): loadModel D/Tflite_audio( 4423): model name is: assets/google_teach_machine_model.tflite I/tflite ( 4423): Initialized TensorFlow Lite runtime. W/native ( 4423): cpu_feature_guard.cc:36 The TensorFlow library was compiled to use SSE instructions, but these aren't available on your machine. F/libc ( 4423): Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xfffffff4 in tid 4423 (e_audio_example), pid 4423 (e_audio_example)

---

Build fingerprint: 'google/sdk_gphone_x86_arm/generic_x86_arm:11/RSR1.201013.001/6903271:userdebug/dev-keys' Revision: '0' ABI: 'x86' Timestamp: 2021-02-17 23:52:01+0100 pid: 4423, tid: 4423, name: e_audio_example >>> tfliteaudio.tflite_audio_example <<< uid: 10153 signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xfffffff4 eax 00000000 ebx aafb0ff4 ecx 000000a0 edx 00000008 edi e85140e8 esi aafba5f8 ebp ff9f37b8 esp ff9f3770 eip a619dd29`

[Tanelo]

When running the decodwave on android (as well on the ios simulator) it's working fine !

Debug session : Launching lib/main.dart on sdk gphone x86 arm in debug mode... lib/main.dart:1 ✓ Built build/app/outputs/flutter-apk/app-debug.apk. Connecting to VM Service at ws://127.0.0.1:56944/F_W8790dsa8=/ws D/Tflite_audio( 4816): loadModel D/Tflite_audio( 4816): model name is: assets/decoded_wav_model.tflite I/tflite ( 4816): Initialized TensorFlow Lite runtime. D/Tflite_audio( 4816): label name is: assets/decoded_wav_label.txt D/Tflite_audio( 4816): Labels: [_silence_, _unknown_, yes, no, up, down, left, right, on, off, stop, [go]](url)

[cmalbuquerque]

@Tanelo you cannot use 'decodedWav' as inputType if you are using a GTM model! Try to change the inputType to 'rawAudio' in your audio recognition method

[Caldarie]

@Tanelo many thanks for the error logs.

It looks like an error with the tensorflow library, though I cant seem to glean any more information than that.

Can you provide more logs from the beginning?

Also, using @cmalbuquerque suggestion, can you check that you are using these parameters:

final String model = 'assets/google_teach_machine_model.tflite';
final String label = 'assets/google_teach_machine_label.txt';
final String inputType = 'rawAudio';
final int sampleRate = 44100;
final int recordingLength = 44032;
final int bufferSize = 22016;
final int numOfInferences = 1;

[Tanelo]

More of the debug mode :

` /Tflite_audio( 5403): loadModel D/Tflite_audio( 5403): model name is: assets/google_teach_machine_model.tflite I/tflite ( 5403): Initialized TensorFlow Lite runtime. W/native ( 5403): cpu_feature_guard.cc:36 The TensorFlow library was compiled to use SSE instructions, but these aren't available on your machine. F/libc ( 5403): Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xfffffff4 in tid 5403 (e_audio_example), pid 5403 (e_audio_example)

---

Build fingerprint: 'google/sdk_gphone_x86_arm/generic_x86_arm:11/RSR1.201013.001/6903271:userdebug/dev-keys' Revision: '0' ABI: 'x86' Timestamp: 2021-02-18 08:51:25+0100 pid: 5403, tid: 5403, name: e_audio_example >>> tfliteaudio.tflite_audio_example <<< uid: 10153 signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xfffffff4 eax 00000000 ebx abdaeff4 ecx 000000a0 edx 0000000d edi e85140e8 esi abdb85f8 ebp ff9f37b8 esp ff9f3770 eip a6f9bd29 backtrace: #00 pc 0039ad29 /data/app/~~1HiIG67KtPY_A-PHL0219w==/tfliteaudio.tflite_audio_example-zsfQjns-rATmoaLaZf6gUw==/lib/x86/libtensorflowlite_flex_jni.so #01 pc 046f57e7 /data/app/~~1HiIG67KtPY_A-PHL0219w==/tfliteaudio.tflite_audio_example-zsfQjns-rATmoaLaZf6gUw==/lib/x86/libtensorflowlite_flex_jni.so #02 pc 046f5462 /data/app/~~1HiIG67KtPY_A-PHL0219w==/tfliteaudio.tflite_audio_example-zsfQjns-rATmoaLaZf6gUw==/lib/x86/libtensorflowlite_flex_jni.so #03 pc 046f5a6b /data/app/~~1HiIG67KtPY_A-PHL0219w==/tfliteaudio.tflite_audio_example-zsfQjns-rATmoaLaZf6gUw==/lib/x86/libtensorflowlite_flex_jni.so #04 pc 04521527 /data/app/~~1HiIG67KtPY_A-PHL0219w==/tfliteaudio.tflite_audio_example-zsfQjns-rATmoaLaZf6gUw==/lib/x86/libtensorflowlite_flex_jni.so #05 pc 00397b75 /data/app/~~1HiIG67KtPY_A-PHL0219w==/tfliteaudio.tflite_audio_example-zsfQjns-rATmoaLaZf6gUw==/lib/x86/libtensorflowlite_flex_jni.so #06 pc 0005f918 /apex/com.android.runtime/bin/linker (_dl__ZL10call_arrayIPFviPPcS1_EEvPKcPT_jbS5+312) (BuildId: c17fda87f98636d6da3d69604bb1486c) #07 pc 0005fbbc /apex/com.android.runtime/bin/linker (__dl__ZN6soinfo17call_constructorsEv+588) (BuildId: c17fda87f98636d6da3d69604bb1486c) #08 pc 00043be2 /apex/com.android.runtime/bin/linker (__dl__Z9do_dlopenPKciPK17android_dlextinfoPKv+2674) (BuildId: c17fda87f98636d6da3d69604bb1486c) #09 pc 0003e4b2 /apex/com.android.runtime/bin/linker (__dl__ZL10dlopen_extPKciPK17android_dl

#145 pc 0036fb02 /apex/com.android.art/lib/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.llvm.16375758241455872412)+370) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #146 pc 00379b00 /apex/com.android.art/lib/libart.so (art::interpreter::EnterInterpreterFromEntryPoint(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*)+176) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #147 pc 0078b325 /apex/com.android.art/lib/libart.so (artQuickToInterpreterBridge+1061) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #148 pc 0014220d /apex/com.android.art/lib/libart.so (art_quick_to_interpreter_bridge+77) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #149 pc 00893656 /system/framework/x86/boot-framework.oat (com.android.internal.os.ZygoteInit.main+2102) (BuildId: 9a9778e61b43d349325d0bb85244bd9bc95ff387) #150 pc 0013baf2 /apex/com.android.art/lib/libart.so (art_quick_invoke_static_stub+418) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #151 pc 001d0392 /apex/com.android.art/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+258) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #152 pc 0062e653 /apex/com.android.art/lib/libart.so (art::JValue art::InvokeWithVarArgsart::ArtMethod*(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, art::ArtMethod*, char*)+579) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #153 pc 0062eb25 /apex/com.android.art/lib/libart.so (art::JValue art::InvokeWithVarArgs<_jmethodID*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, char*)+85) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #154 pc 004ce64f /apex/com.android.art/lib/libart.so (art::JNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, char*)+735) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #155 pc 003f8aae /apex/com.android.art/lib/libart.so (art::(anonymous namespace)::CheckJNI::CallMethodV(char const*, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, char*, art::Primitive::Type, art::InvokeType)+2846) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #156 pc 003e60d9 /apex/com.android.art/lib/libart.so (art::(anonymous namespace)::CheckJNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, char*)+73) (BuildId: 8191579dfafff37a5cbca70f9a73020f) #157 pc 0008f90e /system/lib/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+62) (BuildId: 588f2cd5873ff4273bb25b25edb82606) #158 pc 00098c8e /system/lib/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vectorandroid::String8 const&, bool)+910) (BuildId: 588f2cd5873ff4273bb25b25edb82606) #159 pc 00003804 /system/bin/app_process32 (main+1588) (BuildId: c5eedbfb6130af84c3db8e121fb1202e) #160 pc 000522e3 /apex/com.android.runtime/lib/bionic/libc.so (__libc_init+115) (BuildId: 6e3a0180fa6637b68c0d181c343e6806) Lost connection to device. `

Let me explain you what's happening, when running the app, the debug session is starting, everything is okay, then, at the moment when the app is loading and opening on the android emulator, it's freezing and closing directly. Then you have : 'lost connection to device'

[Caldarie]

Hi @Tanelo

sorry for the late reply. I’ve been a bit busy with another project. If it’s not too much to ask, do you have a deadline? I may need time to investigate this issue

[Tanelo]

@Caldarie I have no deadline, or like a week or two. Thanks !