google_ml_kit_flutter icon indicating copy to clipboard operation
google_ml_kit_flutter copied to clipboard

PlatformException when trying to detect objects on a custom model (3D tensor vs 4D tensor)

Open Theunodb opened this issue 3 years ago • 1 comments

I used Cloud AutoML to train a custom model that suppose to detect marks on a piece of paper. I have the dataset exported as a TFLite file, and i have it hosted on firebase.

I managed to download the file and initiate the objectDetector fine. but are getting an error when processing an input image.

This is how i download the model file

  static Future<File> loadModelFileFromFirebase() async {
    try {
      FirebaseModelDownloader downloader = FirebaseModelDownloader.instance;

      List<FirebaseCustomModel> models = await downloader.listDownloadedModels();
      for (FirebaseCustomModel model in models) {
        print('Name: ${model.name}');
      }

      FirebaseModelDownloadConditions conditions = FirebaseModelDownloadConditions(
        iosAllowsCellularAccess: true,
        iosAllowsBackgroundDownloading: false,
        androidChargingRequired: false,
        androidWifiRequired: false,
        androidDeviceIdleRequired: false,
      );

      FirebaseCustomModel model = await downloader.getModel(
        'xxx',
        FirebaseModelDownloadType.latestModel,
        conditions,
      );

      File modelFile = model.file;

      return modelFile;
    } catch (exception) {
      print('Failed on loading your model from Firebase: $exception');
      print('The program will not be resumed');
      rethrow;
    }
  }

This is how i initialise the object detector:

  static Future<ObjectDetector> initializeLocalDetector(File modelFile, double confidenceThreshold, int maximumLabelsPerObject) async {
    try {
      final options = LocalObjectDetectorOptions(
        mode: DetectionMode.single,
        modelPath: modelFile.path,
        classifyObjects: true,
        multipleObjects: true,
        confidenceThreshold: confidenceThreshold,
        maximumLabelsPerObject: maximumLabelsPerObject,
      );

      return ObjectDetector(options: options);
    } catch (exception) {
      print('Failed on loading your model to the TFLite interpreter: $exception');
      print('The program will not be resumed');
      rethrow;
    }
  }

When i call:

        List<DetectedObject> objects = await state.mainShoddyState.objectDetector!.processImage(inputImage);

I get the following error:

PlatformException(Error 3, com.google.visionkit.pipeline.error, Pipeline failed to fully start:
CalculatorGraph::Run() failed in Run: 
Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1)., null)

Is there something i'm missing?

Theunodb avatar Aug 12 '22 12:08 Theunodb

I am getting the same error when running my custom model. Mine is stored locally. I get the error when running on iPhone 13 pro, ios 15.6.1

jasonJamEther avatar Sep 21 '22 21:09 jasonJamEther