Aruco Java - crash

[westman2000]

System information (version)

• OpenCV => 3.2
• Operating System / Platform => Android/java

Detailed description

I build OpenCV and Aruco module, from Git. Download OpenCV master branch and contrib. copy Aruco folder from contrib to modules folder of opencv and run build script:

cmake \
-DANDROID_ABI=armeabi-v7a \
-DANDROID_NATIVE_API_LEVEL=android-14 \
-DANDROID_SDK=/Users/admin/Library/Android/sdk \
-DANDROID_NDK=/Users/admin/Library/Android/sdk/ndk-bundle \
-DANT_EXECUTABLE=/Users/admin/Documents/apache-ant/bin/ant \
-DJAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home \
-DJAVA_JVM_LIBRARY=/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/include/jni.h \
-DJAVA_INCLUDE_PATH=/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/include \
-DBUILD_FAT_JAVA_LIB=ON \
-DBUILD_SHARED_LIBS=OFF \
-DCMAKE_TOOLCHAIN_FILE=../android/android.toolchain.cmake $@ ../..

everything went well. i include libraries and generated java classes in Android Studio. OpenCV is working, I tried several functions and it works. But with Aruco something is wrong, it always crash on Aruco.detectMarkers(...)

Generated java code:

public static void detectMarkers(Mat image, Dictionary dictionary, List<Mat> corners, Mat ids)
{
   Mat corners_mat = new Mat();
   detectMarkers_1(image.nativeObj, dictionary.nativeObj, corners_mat.nativeObj, ids.nativeObj);
   Converters.Mat_to_vector_Mat(corners_mat, corners);
   corners_mat.release();
   return;
}

I create my wraper around detectMarkers() almost like generated code and it works but without parameter OutputArrayOfArrays corners and it works. If i add this parameter, my function will look like generated and then i see error:

OpenCV Error: The function/feature is not implemented (Only Mat vector, UMat vector, and vector<vector> OutputArrays are currently supported.) in void cv::aruco::_copyVector2Output(std::vector<std::vector<cv::Point_<float> > >&, cv::OutputArrayOfArrays), file /Users/admin/Downloads/opencv-3.2.0/modules/aruco/src/aruco.cpp, line 595

so my guess is why it is impossible to use the generated code - because bad conversion between java Mat and C++ OutputArrayOfArrays

[mshabunin]

@westman2000 , can you please provide a minimal standalone reproducing example?

[westman2000]

Here it is: https://github.com/westman2000/OpenCVSample (Checkout and open project in Android Studio) This is with prebuild libraries, if you will need i can create with JNI support/wrapper

[psikharpax]

+1, I Have the same Problem @mshabunin

[44326334]

+1 I have the same issue !!! @mshabunin

[chaoyangnz]

+1 +1 +1

[chaoyangnz]

@westman2000 could you share your own wrap code?

[westman2000]

Sorry, but I'm not working on this project anymore.

[chaoyangnz]

Thanks. I rewraped the function by myself, like the following code, and it works.

extern "C"
JNIEXPORT void JNICALL
Java_com_a3dprintingsystems_robopose_CvAruco_detectMarkers_11(JNIEnv *env, jclass type,
                                                           jlong imgAddr, jlong dictionaryAddr,
                                                           jlong cornersAddr, jlong idsAddr) {

    cv::Mat* image = (cv::Mat*)imgAddr;
    Ptr<aruco::Dictionary> dictionary = *((Ptr<aruco::Dictionary>*)(dictionaryAddr));
    Ptr<aruco::DetectorParameters> detectorParams = aruco::DetectorParameters::create();

    vector< int > ids;
    vector< vector< Point2f > > corners, rejected;
    vector< Vec3d > rvecs, tvecs;

    aruco::detectMarkers(*image, dictionary, corners, ids, detectorParams, rejected);

    aruco::drawDetectedMarkers(*image, corners, ids, Scalar(255, 255, 0));

    vector_int_to_Mat(ids, *((cv::Mat*)idsAddr));
    vector_vector_Point2f_to_Mat(corners, *((cv::Mat*)cornersAddr));
}

And in Android, use it like this:

Mat ids = new Mat();
List<Mat> corners = new ArrayList<>();
CvAruco.detectMarkers(grayFrame, Aruco.getPredefinedDictionary(DICT_4X4_50), corners, ids);

Hope it helps

[Kafakk]

could someone help me, please. I'm a newbie Android Studio developer. Now I do the project about detecting the Aruco marker on Android. I try to import opencv3-android-sdk-with-contrib from https://github.com/richdyang/opencv3-android-sdk-with-contrib to Android Studio project, but I got this error message "Cannot resolve corresponding JNI function" from any .java class in opencv module. I already add jniLibs folder to the main folder. and this is my openCV build.gradle If I do something wrong please tell me.

[chaoyangnz]

@Kafakk Ignore these error message, they won't influence the build. Just build the App.

These errors are there because the JNI source code is not in your project. You don't need them actually because they have been compiled into libopencv_aruco.a

[Kafakk]

@richdyang Thank you. Now I can run my project as well.

I try to detect marker, but the ids of the marker are Mat type. I have another question is it possible to collect ids of marker type of integer like it c++.

[chaoyangnz]

@Kafakk You can:

Mat ids = ... ; 
int[] idArray = ids.toArray(); 
List<Integer> idList = ids.toList();

Then do whatever you'd like to do.

A tip: in OpenCV, you will see Mat everywhere, but you still need to be aware of the actual type. Mat is just a container, you should be concerned with the underlying data type and channels. For ids Aruco returns, it's 32SC1 with 4 rows x 1 column.

[Kafakk]

@richdyang Well, I try to do but in the Mat I does not see the toArray() function. I want to collect the ids of markers, but in the Log the Mat that I collected is:

11-16 00:11:47.273 31268-31399/com.example.dream.letbyread D/MainActivity: ids is: Mat [ 11CV_32SC1, isCont=true, isSubmat=false, nativeObj=0x7020b93ee0, dataAddr=0x701fff7300 ] 11-16 00:11:47.319 31268-31399/com.example.dream.letbyread D/MainActivity: ids is: Mat [ 11CV_32SC1, isCont=true, isSubmat=false, nativeObj=0x7020b92e00, dataAddr=0x701fff7500 ] 11-16 00:11:47.363 31268-31399/com.example.dream.letbyread D/MainActivity: ids is: Mat [ 11CV_32SC1, isCont=true, isSubmat=false, nativeObj=0x7020b92f80, dataAddr=0x701fff7140 ] 11-16 00:11:47.405 31268-31399/com.example.dream.letbyread D/MainActivity: ids is: Mat [ 11CV_32SC1, isCont=true, isSubmat=false, nativeObj=0x7020b93700, dataAddr=0x701fff7100 ]

I don't know what is the index of the marker. And this is my onCameraFrame function:

public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame){
        Mat ids = new Mat();
        List<Mat> corners = new ArrayList<>();
        Dictionary dictionary = Aruco.getPredefinedDictionary(Aruco.DICT_4X4_250);
        Aruco.detectMarkers(inputFrame.gray(), dictionary, corners, ids);
        Log.d(TAG,"ids is: "+ids);
        mRgba = inputFrame.rgba();
        return mRgba;
    }

Another question. could you explain about your c++ rewrapped the function. How it links to the java class, and how you import the Aruco library.

[chaoyangnz]

No need to rewrap now. Because I rebuild OpenCV Android SDK with Aruco, it now works.

You should write like this if you are more familiar to use array:

int[] idsArr = ids.toArray();
Log.d(TAG, "id 0: " + idsArr[0]);

[Kafakk]

I write it like this but it is not found toArray() function.

[chaoyangnz]

@Kafakk declare the ids like this:

MatOfInt ids = new MatOfInt();

[Kafakk]

@richdyang Now I can detect the marker on the android phone. Thank you very much.