clip.cpp icon indicating copy to clipboard operation
clip.cpp copied to clipboard

Add Android sample (JNI binding + Java wrapper class)

Open shubham0204 opened this issue 1 year ago • 0 comments

This PR adds examples/clip.android, an Android sample that interfaces with the shared libraries of clip.cpp through JNI. The structure of the project is highly inspired from llama.cpp/examples/llama.android.

Summary of changes

  • examples/clip.android is an Android project with two modules app and clip.
  • The app module contains the sources for a demo application which allows the user to select an image and enter a text description to compare them (uses Kotlin/Compose)
  • The clip module contains clip/src/main/cpp/clip-android that contains the JNI interfaces which allow communication with the underlying functions from clip.h.
  • Also, the clip module contains clip/src/main/java/android/clip/cpp/CLIPAndroid.java which is wrapper class containing native methods.
  • The clip module also houses an instrumented test clip/src/androidTest/java/android/example/clip/CLIPInstrumentedTest.kt written in Kotlin
  • A README.md is included in the examples/clip.android directory which describes how to run the sample app and the instrumented test
  • clip.cpp is modified to address issue #99
  • CMakeLists.txt is modified to add -mcpu=native when ${CMAKE_SYSTEM_PROCESSOR} MATCHES "aarch64" only if -DCLIP_NATIVE is enabled
  • added a new method clip_image_preprocess_no_resize in clip.h and clip.cpp which does works exactly like clip_image_preprocess but does not perform linear interpolation (no image resizing). Android's Bitmap class and Python's PIL can handle bilinear interpolation easily, so it would be great to have a method that skips resizing.

[!NOTE] Though the clip module here in built as an Android library module, it can be built into a 'Java library' module (packaged as a JAR) and used in other Java-only projects as a port to clip.cpp

It was fun working on this sample, and I would like @monatis to review it and share feedback.

shubham0204 avatar Sep 14 '24 05:09 shubham0204