Cannot use on custom model

[hallowcard13]

Tried to use my custom model, But when i try to hit the Scan button the app crashes. Do you have a detailed instructions on how to use custom model ?

[devbridie]

Do you have any specific error messages? Try these steps:

1. Download your custom model (I've tried "inception v4_1") and place it in app/src/main/res/assets.
2. Uncomment lines in MLKitObjectDetector to specify the model.

[bacnv-1260]

@hallowcard13 @devbridie hi any solution resolved?

[tbikash62]

Hello @devbridie

Thank you for follow ups.

I have trained a custom model. But since very long tried to integrate it. Unfortunately it's failing.

Error, it says -
E/AndroidRuntime: FATAL EXCEPTION: DefaultDispatcher-worker-1 Process: com.google.ar.core.examples.java.ml, PID: 2794 com.google.mlkit.common.MlKitException: Failed to initialize detector. Unexpected number of dimensions for output index 1: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1). at com.google.mlkit.vision.vkp.PipelineManager.start(com.google.mlkit:vision-internal-vkp@@18.0.0:63) at com.google.mlkit.vision.objects.custom.internal.zzg.tryLoad(com.google.mlkit:object-detection-custom@@16.3.1:3) at com.google.mlkit.common.sdkinternal.model.CustomModelLoader.load(com.google.mlkit:common@@17.1.1:3) at com.google.mlkit.vision.objects.custom.internal.zzh.load(com.google.mlkit:object-detection-custom@@16.3.1:2) at com.google.mlkit.common.sdkinternal.ModelResource.zza(Unknown Source:18) at com.google.mlkit.common.sdkinternal.zzn.run(Unknown Source:10) at com.google.mlkit.common.sdkinternal.zzp.run(com.google.mlkit:common@@17.1.1:2) at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zze(com.google.mlkit:common@@17.1.1:4) at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzc(Unknown Source:8) at com.google.mlkit.common.sdkinternal.zzj.run(Unknown Source:2) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzd(Unknown Source:10) at com.google.mlkit.common.sdkinternal.zzk.run(Unknown Source:2) at java.lang.Thread.run(Thread.java:923) D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning... I/Process: Sending signal. PID: 2794 SIG: 9

Custom model trained using - https://www.tensorflow.org/lite/api_docs/python/tflite_model_maker/object_detector/EfficientDetSpec

I understand, ML kit wants to run a model with batch size one. But also if I have batch size one too, it throws this error. So to clear up things, I tried tensorflow debugging using few commands to get the tensors. Still it's not clear to me what needs to be changed. I'm attaching a tflite file for your testing purpose.

It will be great if you can test it out. Also please let me know, if you need any further detailed information. I'm happy to let you know in the debugging process.

model.zip

[GiovanniTurri]

@hallowcard13 @devbridie @tbikash62 The error is still present. Any news?

I've tried other .tflite models but the error is the same. In the next few days I'll try to mix up the actual sample code with https://github.com/googlesamples/mlkit/blob/2f80c301bae79136efa9096881b6e44bee7866a3/android/android-snippets/app/src/main/java/com/google/example/mlkit/kotlin/ObjectDetectionActivity.kt and https://github.com/googlesamples/mlkit/blob/4be2fe856e0b3fd95af370fe5413a33494698067/android/android-snippets/app/src/main/java/com/google/example/mlkit/kotlin/MLKitVisionImage.kt#L93-L118

Hope to update this comment with a solution :) Thank you

P.S. I also noticed a complete freeze changing setDetectorMode(CustomObjectDetectorOptions.STREAM_MODE) instead of SINGLE_IMAGE_MODE by default