tensorflow icon indicating copy to clipboard operation
tensorflow copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

Enable NNAPI Crash

Open Whrothus opened this issue 2 years ago • 6 comments

System information

  • Xamarin.Android Android 12.0
  • Xamarin Tensorflow Lite 2.6.0.1
  • Visual Studio 2022 c#

Standalone code to reproduce the issue

Enableing NNAPI (options.SetUseNNAPI(true);) interpreter = new Interpreter(mappedByteBuffer, options); https://tfhub.dev/tensorflow/lite-model/ssd_mobilenet_v1/1/metadata/1?lite-format=tflite That works

all my trained models Crash Work when i don't Enable NNAPI (I Comment options.SetUseNNAPI(true) but its should be slower)

Even when i take http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8.tar.gz

tflite_directory = '/content/tfliteExport' pipeline_file = '/content/models/research/deploy/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/pipeline.config'

last_model_path = '/content/models/research/deploy/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/checkpoint' !python /content/models/research/object_detection/export_tflite_graph_tf2.py
--pipeline_config_path={pipeline_file}
--trained_checkpoint_dir={last_model_path}
--output_directory={tflite_directory} import cv2

image_path = '/content/gdrive/MyDrive/Map/foto/Test'

image_path = '/content/img'

def representative_dataset_gen(): for f_name in os.listdir(image_path): file_path = os.path.normpath(os.path.join(image_path, f_name)) print(file_path) img = cv2.imread(file_path) img = cv2.resize(img, (640,640)) img = img / 255.0 img = np.reshape(img, (1, 640, 640, 3)) image = img.astype(np.float32) yield [image] converter = tf.lite.TFLiteConverter.from_saved_model('/content/tfliteExport/saved_model') converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.representative_dataset = representative_dataset_gen converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8] converter.inference_input_type = tf.uint8 converter.inference_output_type = tf.float32

converter.inference_input_type = tf.uint8

converter.inference_output_type = tf.uint8

tflite_model = converter.convert() with open('{}/TFmodelV3.tflite'.format('/content/tfliteExport/saved_model'), 'wb') as w: w.write(tflite_model) print("tflite convert complete! - {}/TFmodelV3.tflite".format('/content/newTFLite/saved_model')) Any other info / logs {Java.Lang.IllegalArgumentException: Internal error: Failed to apply delegate: NN API returned error ANEURALNETWORKS_BAD_DATA at line 1053 while adding operation.

Restored original execution plan after delegate application failure. at Java.Interop.JniEnvironment+InstanceMethods.CallNonvirtualVoidMethod (Java.Interop.JniObjectReference instance, Java.Interop.JniObjectReference type, Java.Interop.JniMethodInfo method, Java.Interop.JniArgumentValue* args) [0x00088] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniEnvironment.g.cs:12324 at Java.Interop.JniPeerMembers+JniInstanceMethods.FinishCreateInstance (System.String constructorSignature, Java.Interop.IJavaPeerable self, Java.Interop.JniArgumentValue* parameters) [0x0003e] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniInstanceMethods.cs:142 at Xamarin.TensorFlow.Lite.Interpreter..ctor (Java.Nio.ByteBuffer byteBuffer, Xamarin.TensorFlow.Lite.Interpreter+Options options) [0x0009d] in :0 at CameraTF.TensorflowLiteService.Initialize () [0x00017] in X:\Snelle TF Xam\CameraTF-master\src\CameraTF\AR\TensorflowLiteService.cs:62 --- End of managed Java.Lang.IllegalArgumentException stack trace --- java.lang.IllegalArgumentException: Internal error: Failed to apply delegate: NN API returned error ANEURALNETWORKS_BAD_DATA at line 1053 while adding operation.

Restored original execution plan after delegate application failure. at org.tensorflow.lite.NativeInterpreterWrapper.applyDelegate(Native Method) at org.tensorflow.lite.NativeInterpreterWrapper.applyDelegates(NativeInterpreterWrapper.java:486) at org.tensorflow.lite.NativeInterpreterWrapper.init(NativeInterpreterWrapper.java:88) at org.tensorflow.lite.NativeInterpreterWrapper.(NativeInterpreterWrapper.java:66) at org.tensorflow.lite.Interpreter.(Interpreter.java:247) at crc649b31f1000968ddb3.MainActivity.n_onCreate(Native Method) at crc649b31f1000968ddb3.MainActivity.onCreate(MainActivity.java:31) at android.app.Activity.performCreate(Activity.java:7994) at android.app.Activity.performCreate(Activity.java:7978) at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1309) at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3422) at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3601) at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85) at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2066) at android.os.Handler.dispatchMessage(Handler.java:106) at android.os.Looper.loop(Looper.java:223) at android.app.ActivityThread.main(ActivityThread.java:7656) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947) }

Whrothus avatar Jul 27 '22 19:07 Whrothus

@sachinprasadhs if you need help just let me know

Whrothus avatar Aug 03 '22 12:08 Whrothus

@Whrothus , Apologies for the delayed response, I'm looking into your issue. I will get back to you if I need any details from you. Thanks

sachinprasadhs avatar Aug 03 '22 17:08 sachinprasadhs

Could you try the option to setUseNnapiCpu to true in the NnApiDelegate.Options object and let us know if it is still crashing.

sachinprasadhs avatar Aug 03 '22 17:08 sachinprasadhs

So i did Interpreter.Options options = new Interpreter.Options(); options.SetUseNNAPI(true); NnApiDelegate.Options NnApiOption = new NnApiDelegate.Options(); NnApiOption.SetUseNnapiCpu(true); NnApiDelegate nnApiDelegate = new NnApiDelegate(NnApiOption); options.AddDelegate(nnApiDelegate); try { interpreter = new Interpreter(mappedByteBuffer, options);

.TFlite = https://drive.google.com/file/d/1RZfTMnxOdKnDyBUBFb0kk4EJoxdUGlXr/view?usp=sharing

Crash

{Java.Lang.IllegalArgumentException: Internal error: Failed to apply delegate: NN API returned error ANEURALNETWORKS_BAD_DATA at line 1053 while adding operation.

Node number 158 (TfLiteNnapiDelegate) failed to prepare.

Restored original execution plan after delegate application failure. at Java.Interop.JniEnvironment+InstanceMethods.CallNonvirtualVoidMethod (Java.Interop.JniObjectReference instance, Java.Interop.JniObjectReference type, Java.Interop.JniMethodInfo method, Java.Interop.JniArgumentValue* args) [0x00088] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniEnvironment.g.cs:12324 at Java.Interop.JniPeerMembers+JniInstanceMethods.FinishCreateInstance (System.String constructorSignature, Java.Interop.IJavaPeerable self, Java.Interop.JniArgumentValue* parameters) [0x0003e] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniInstanceMethods.cs:142 at Xamarin.TensorFlow.Lite.Interpreter..ctor (Java.Nio.ByteBuffer byteBuffer, Xamarin.TensorFlow.Lite.Interpreter+Options options) [0x0009d] in :0 at CameraTF.TensorflowLiteService.Initialize () [0x00034] in X:\Snelle TF Xam\CameraTF-master\src\CameraTF\AR\TensorflowLiteService.cs:68 --- End of managed Java.Lang.IllegalArgumentException stack trace --- java.lang.IllegalArgumentException: Internal error: Failed to apply delegate: NN API returned error ANEURALNETWORKS_BAD_DATA at line 1053 while adding operation.

Node number 158 (TfLiteNnapiDelegate) failed to prepare.

Restored original execution plan after delegate application failure. at org.tensorflow.lite.NativeInterpreterWrapper.applyDelegate(Native Method) at org.tensorflow.lite.NativeInterpreterWrapper.applyDelegates(NativeInterpreterWrapper.java:480) at org.tensorflow.lite.NativeInterpreterWrapper.init(NativeInterpreterWrapper.java:88) at org.tensorflow.lite.NativeInterpreterWrapper.(NativeInterpreterWrapper.java:66) at org.tensorflow.lite.Interpreter.(Interpreter.java:247) at crc649b31f1000968ddb3.MainActivity.n_onCreate(Native Method) at crc649b31f1000968ddb3.MainActivity.onCreate(MainActivity.java:31) at android.app.Activity.performCreate(Activity.java:7994) at android.app.Activity.performCreate(Activity.java:7978) at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1309) at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3422) at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3601) at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85) at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2066) at android.os.Handler.dispatchMessage(Handler.java:106) at android.os.Looper.loop(Looper.java:223) at android.app.ActivityThread.main(ActivityThread.java:7656) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947) }

Whrothus avatar Aug 03 '22 22:08 Whrothus

Hi @Whrothus, thanks for reporting the issue. Could you share the following information to help us analyze the problem?

  • Which device were you using when the issue occured?
  • What is the Android build number / version of that device?

miaowang14 avatar Aug 09 '22 23:08 miaowang14

@miaowang14 One plus 7 Oxygen 10.0.9.1 and pixel 5 emulator. sdk_gphone_x86_64_arm64-userdebug 11 RSR1.210722.013 7800151 dev-keys. Is there a emulator you prefer ?

Whrothus avatar Aug 10 '22 00:08 Whrothus

@multiverse-tf Do you need something else ?

Whrothus avatar Aug 16 '22 13:08 Whrothus

@miaowang14 Do you have a emulator you prefer ?

Whrothus avatar Aug 25 '22 20:08 Whrothus

For both the OnePlus 7 and the emulator, could you try adb shell setprop debug.nn.vlog 1 and rerun your test and share the logcat through adb logcat?

Also, I just realized that you were using Xamarin Tensorflow Lite 2.6.0.1, which seems to be out of date regarding NNAPI delegate implementation. I wonder if you have tried to use the latest official TFLite packages to run your models? E.g. use the TFLite benchmark app (https://www.tensorflow.org/lite/performance/measurement):

adb shell am start -S \
  -n org.tensorflow.lite.benchmark/.BenchmarkModelActivity \
  --es args '"--graph=/data/local/tmp/your_model.tflite \
              --use_nnapi=true --nnapi_allow_fp16=true"'

miaowang14 avatar Sep 07 '22 03:09 miaowang14

Think i f up att converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]

Whrothus avatar Sep 23 '22 17:09 Whrothus

Is there any way you could include the select ops as well in your converter and check the outcome.

converter.target_spec.supported_ops = [
  tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
  tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]

sachinprasadhs avatar Sep 23 '22 18:09 sachinprasadhs

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

google-ml-butler[bot] avatar Sep 30 '22 18:09 google-ml-butler[bot]

Not resolved but cant make the "adb shell" Command to work

Whrothus avatar Oct 05 '22 15:10 Whrothus

Hi,

Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.

The TFLite team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.

Please follow the release notes to stay up to date with the latest developments which are happening in the TFLite space.

Thanks.

pkgoogle avatar Sep 27 '23 20:09 pkgoogle

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

github-actions[bot] avatar Oct 05 '23 01:10 github-actions[bot]

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.

github-actions[bot] avatar Oct 13 '23 01:10 github-actions[bot]

Are you satisfied with the resolution of your issue? Yes No

google-ml-butler[bot] avatar Oct 13 '23 01:10 google-ml-butler[bot]