Xamarin.Google.MLKit.Vision.Objects.Defaults.ObjectDetectorOptions.Builder has no required public methods from original Java class

[mmarinchenko]

Xamarin.Android Version:

11.1.0.26 (Visual Studio Community 2019 v16.8.5)

Operating System & Version:

Windows 10 Home Single Language 20H2 v19042.804

Google Play Services Version

117.6.0 with AndroidX dependencies

Relevant information

Packages used:

    <Reference Include="Mono.Android" />
    <Reference Include="System" />

    <PackageReference Include="Xamarin.Google.Android.Material">
      <Version>1.3.0</Version>
    </PackageReference>
    <PackageReference Include="Xamarin.Google.MLKit.ObjectDetection">
      <Version>116.2.0</Version>
    </PackageReference>

Google ML Kit Object Detection Guide with Kotlin and Java samples:

• https://developers.google.com/ml-kit/vision/object-detection/android

Related Google ML Kit docs:

• https://developers.google.cn/android/reference/com/google/mlkit/vision/objects/ObjectDetectorOptionsBase
• https://developers.google.cn/android/reference/com/google/mlkit/vision/objects/defaults/ObjectDetectorOptions
• https://developers.google.cn/android/reference/com/google/mlkit/vision/objects/defaults/ObjectDetectorOptions.Builder

Minimal Repro Code Sample

Get the following Java code from Google ML Kit Object Detection Guide (link provided above):

// Multiple object detection in static images
ObjectDetectorOptions options =
        new ObjectDetectorOptions.Builder()
                .setDetectorMode(ObjectDetectorOptions.SINGLE_IMAGE_MODE)
                .enableMultipleObjects()
                .enableClassification()  // Optional
                .build();

and write it in C#:

using Xamarin.Google.MLKit.Vision.Objects.Defaults;
<...>
// Multiple object detection in static images
var options = new ObjectDetectorOptions.Builder()
        .SetDetectorMode(ObjectDetectorOptions.SingleImageMode)
        .EnableMultipleObjects()
        .EnableClassification()  // Optional
        .Build();

This will not compile because all Builder's methods are missing, except Build().

[mmarinchenko]

I tried to overcome this by implementing a subclass of ObjectDetectorOptionsBase which has its own nested protected internal abstract class Builder with required methods.

Result looks odd... I had to implement 2 nested classes for that:

• one as private implementation of protected abstract Builder which required for ObjectDetectorOptionsBase constructor
• and another one as public wrapper to expose API to consumers of new subclass

DetectorOptions.cs (click to expand)

    public sealed class DetectorOptions : ObjectDetectorOptionsBase  // Java.Lang.Object type
    {
        private DetectorOptions(BaseBuilder baseBuilder) : base(baseBuilder) { }

        private sealed class BaseBuilder : ObjectDetectorOptionsBase.Builder  // Java.Lang.Object type
        {
            private readonly Builder _builder;

            public BaseBuilder(Builder builder) => _builder = builder;

            protected override ObjectDetectorOptionsBase Build() => _builder.Build();
        }

        new public sealed class Builder  // pure managed type, hides base.Builder
        {
            private readonly BaseBuilder _baseBuilder;

            private readonly DetectorOptions _detectorOptions;

            public Builder()
            {
                _baseBuilder = new BaseBuilder(this);
                _detectorOptions = new DetectorOptions(_baseBuilder);
            }

            public Builder SetDetectorMode(int detectorMode)
            {
                _baseBuilder.SetDetectorMode(detectorMode);
                return this;
            }

            public Builder SetExecutor(IExecutor executor)
            {
                _baseBuilder.SetExecutor(executor);
                return this;
            }

            public Builder EnableClassification()
            {
                _baseBuilder.EnableClassification();
                return this;
            }

            public Builder EnableMultipleObjects()
            {
                _baseBuilder.EnableMultipleObjects();
                return this;
            }

            public DetectorOptions Build() => _detectorOptions;
        }
    }

So now I can write this in C# and it compiles fine:

var options = new DetectorOptions.Builder()
        .SetDetectorMode(ObjectDetectorOptionsBase.SingleImageMode)
        .EnableMultipleObjects()
        .EnableClassification()  // Optional
        .Build();

But unfortunately this doesn't compile in Java due to DetectorOptions.BaseBuilder is translated into a separate public class which does not have access to the nested protected ObjectDetectorOptionsBase.Builder:

Output (click to expand)

1>  2 errors
1>...\DetectorOptions_BaseBuilder.java(5,67): javac.exe error JAVAC0000:  error: Builder has protected access in ObjectDetectorOptionsBase
1>...\DetectorOptions_BaseBuilder.java(5,67): javac.exe error JAVAC0000: 	extends com.google.mlkit.vision.objects.ObjectDetectorOptionsBase.Builder
1>...\DetectorOptions_BaseBuilder.java(5,67): javac.exe error JAVAC0000: 
1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000:  error: cannot find symbol
1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000: 		if (getClass () == DetectorOptions_BaseBuilder.class)
1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000:   symbol:   method getClass()
1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000:   location: class DetectorOptions_BaseBuilder

DetectorOptions_BaseBuilder.java (click to expand)

package crc641d15e8c705d2fdbc;


public class DetectorOptions_BaseBuilder
	extends com.google.mlkit.vision.objects.ObjectDetectorOptionsBase.Builder  // first compilation error
	implements
		mono.android.IGCUserPeer
{
/** @hide */
	public static final String __md_methods;
	static {
		__md_methods = 
			"n_build:()Lcom/google/mlkit/vision/objects/ObjectDetectorOptionsBase;:GetBuildHandler\n" +
			"";
		mono.android.Runtime.register ("MSSLink.CameraApp.Detection.DetectorOptions+BaseBuilder, MSSLink.CameraApp", DetectorOptions_BaseBuilder.class, __md_methods);
	}


	public DetectorOptions_BaseBuilder ()
	{
		super ();
		if (getClass () == DetectorOptions_BaseBuilder.class)  // second compilation error
			mono.android.TypeManager.Activate ("MSSLink.CameraApp.Detection.DetectorOptions+BaseBuilder, MSSLink.CameraApp", "", this, new java.lang.Object[] {  });
	}


	public com.google.mlkit.vision.objects.ObjectDetectorOptionsBase build ()
	{
		return n_build ();
	}

	private native com.google.mlkit.vision.objects.ObjectDetectorOptionsBase n_build ();

	private java.util.ArrayList refList;
	public void monodroidAddReference (java.lang.Object obj)
	{
		if (refList == null)
			refList = new java.util.ArrayList ();
		refList.add (obj);
	}

	public void monodroidClearReferences ()
	{
		if (refList != null)
			refList.clear ();
	}
}

Is there a way to translate a nested Java.Lang.Object type written in C# to a nested class in Java while preserving the access modifiers?

[AlexDevMobile]

@mmarinchenko Hi, did you find a solution for this?

[mmarinchenko]

@AlerzDev Hi! I have 2 issues here:

• missing methods in some Builder classes (original one)
• generating a separate public ACW for a nested managed class which inherits a protected nested java class (and therefore could not be un-nested from descendant) - see my comment above.

I'm not sure it's right to discuss second issue here. Maybe it should be moved to Xamarin.Android repo. Anyway I don't have a solution for this.

---

As for original issue I assume the following bindings are incomplete:

• image-labeling
• image-labeling-automl
• image-labeling-custom
• object-detection
• object-detection-custom

Each Transforms/Metadata.xml file in these directories has custom binding rules and the following (or similar) comment:

    <!--
    TODO: possible problems! return type covariance/contravariance (needs investigation)
    
    removing `build` with return="<...>"
    renaming in Addditions
    -->

but only one of corresponding Additions/***Options.cs files (the image-labeling-automl one) actually implements some logic.

So I'm waiting for a comment from @moljac. Or from someone else who can bring this to light :)

[moljac]

@AlerzDev

Thanks for the feedack

missing methods in some Builder classes (original one)

That happens for various reasons (difference between java and c# idioms, generics implementations,...). Even our tooling and our/my mistakes.

I rushed to publish MLKit and to catch up with google, so I might have made some mistakes.

Furthermore - the quality of bindings cannot be determined w/o samples or API testing (unit test). Are you getting where am I heading? The more minimal sample you guys can provide us - the better.

*Options classes were tough because of inheritance and covariance (return types). I need to dive in and check and am not sure when I'll be ready to do so. Namely I need to do tons of updates in XamarinComponents which fix several bugs and are crucial for new updates of AndroidX and GPS-FB-MLKit

[mmarinchenko]

@moljac

Hi! I guess you answered the wrong person :) Anyway thanks for the explanation.

I spent some time playing with build process lately (PR #437). I'll try to write sample which cover my needs in separate PR.

[moljac]

@mmarinchenko

I'll need to dive deeper into the bindings, but sample would help me a lot.

Bindings are important, but samples are also important to verify the bindings.

I finished tons of updates in XamarinComponents and want to fix something from months ago, before I can make further steps (updates, samples and tooliing)

[tomkulaga]

Did you guys have any success with getting the builder options working?

[mmarinchenko]

@tomkulaga

Sorry, I'm busy with another project right now. @moljac is busy too, I guess.

[tomkulaga]

@mmarinchenko

no problems. Did you every try and compile from source and edit the Additions.cs like the image automl?

I’m trying that but keep getting build errors on latest branch and latest tagged release

[mmarinchenko]

Xamarin.Android v11.2 (VS v16.9.*) has issues related to Kotlin metadata (ex. see https://github.com/xamarin/java.interop/issues/826). But I haven't tried VS v16.10 yet.

[tomkulaga]

I had mode success in 16.10 than 16.11 preview.

It finished the binderate target but bailed on the line.

I opened the generated SLN but it failed on an Android generator step late in the build. I was building MLKIT Obejct Detection and got 10/12 built.

Gave up after that :(

Sent from my iPhone

On 1 Jun 2021, at 4:09 pm, Maxim Marinchenko @.***> wrote:

 Xamarin.Android v11.2 (VS v16.9.*) has issues related to Kotlin metadata (ex. see xamarin/java.interop#826). But I haven't tried VS v16.10 yet.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[alexgg94]

Any update on this?

[tomkulaga]

I was able to build them BUT couldn't fix the original issue, so gave up.

[mmarinchenko]

For those who may hit this issue and can't wait for Google MLKit fix in Xamarin.

ONNX can be used for ML tasks on mobile platforms as an alternative.

Blog post: https://devblogs.microsoft.com/xamarin/machine-learning-in-xamarin-forms-with-onnx-runtime/ Sample: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/mobile/examples/Xamarin

[tomkulaga]

Were you able to get realtime object detection working with Onnx?

[mmarinchenko]

No, I haven't tried it yet.