ai.djl.repository.zoo.ModelNotFoundException: ModelZoo doesn't support specified engine: PyTorch

[edgargao]

Description

I will use djl SDK in Android Project.If app is a general application,it work right.If app is a system application,it will throw a excepiton,as this "ai.djl.repository.zoo.ModelNotFoundException: ModelZoo doesn't support specified engine: PyTorch".

Expected Behavior

It can load PyTorch engine.

Error Message

ai.djl.repository.zoo.ModelNotFoundException: ModelZoo doesn't support specified engine: PyTorch

How to Reproduce?

script is: Criteria criteria = Criteria.builder() .optApplication(Application.CV.IMAGE_CLASSIFICATION) .setTypes(NDList.class, NDList.class) .optModelUrls("https://djl-ai.s3.amazonaws.com/resources/demo/pytorch/doodle_mobilenet.zip") .optEngine("PyTorch") .build(); ZooModel model = ModelZoo.loadModel(criteria);

AndroidManifest.xml is: android:sharedUserId="android.uid.system" <manifest xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" package="xxx" android:installLocation="auto" android:sharedUserId="android.uid.system"> <application android:name=".application.App" android:allowBackup="true" android:configChanges="orientation|keyboardHidden" android:debuggable="false" android:hardwareAccelerated="true" android:icon="@drawable/icon" android:label="@string/app_name" android:largeHeap="true" android:persistent="true" android:roundIcon="@drawable/icon" android:supportsRtl="true" tools:ignore="GoogleAppIndexingWarning,HardcodedDebugMode"> <activity android:name=".HomeActivity" android:launchMode="singleInstance">

Steps to reproduce

(Paste the commands you ran that produced the error.)

1.ModelZoo.loadModel(criteria)

What have you tried to solve it?

1.If app is a general application,it work right

Environment Info

Please run the command ./gradlew debugEnv from the root directory of DJL (if necessary, clone DJL first). It will output information about your system, environment, and installation that can help us debug your issue. Paste the output of the command below: Android8.1

PASTE OUTPUT HERE

[frankfliu]

@delgadofarid

I don't have a rooted phone to test such app. Would you please provide stacktrace where the engine loading is failed?

[delgadofarid]

@frankfliu - did you mean to tag @edgargao ?

[frankfliu]

@frankfliu - did you mean to tag @edgargao ?

Sorry, auto-complete, :)

[edgargao]

@delgadofarid

I don't have a rooted phone to test such app. Would you please provide stacktrace where the engine loading is failed?

Hi,frankfliu.It‘s a user-defined exception,didn't have stacktrace message.It's happed in Criteria.java-loadModel() function.

[JohnDoll2023]

@edgargao What computer OS are you using? If you have mac are you on M1?

[edgargao]

@JohnDoll2023 It's not a computer.I worked in an android system app.

[JohnDoll2023]

@edgargao Right, but are you developing this android system app with a windows or mac machine?

[edgargao]

@edgargao Right, but are you developing this android system app with a windows or mac machine?

@JohnDoll2023 I'm developing in windows 11.

[frankfliu]

@edgargao Can you enable debug level, and collect logs? There should be logs related PyTorch engine loading, I suspect either failed to create cached folder or load shared library failed.

Since we don't have a rooted phone, it's hard for us to reproduce your issue.

[xyang16]

@edgargao

We use ServiceLoader to find Engine in the classpath (https://github.com/deepjavalibrary/djl/blob/master/api/src/main/java/ai/djl/engine/Engine.java#L64). This doesn't seems to work for system app.

A workaround for this is to manually register the Engine before loading a model.

1. Add dependency in build.gradle.

dependencies {
    implementation 'androidx.appcompat:appcompat:1.4.2'
    implementation 'ai.djl:api:0.18.0'
    implementation 'ai.djl.android:core:0.18.0'
    implementation 'ai.djl.pytorch:pytorch-engine:0.18.0'
    runtimeOnly 'ai.djl.android:pytorch-native:0.18.0'
}

2. Add a static block in your code.

    static {
        PtEngineProvider provider = new PtEngineProvider();
        Engine.registerEngine(provider);
    }

3. Install the app. Make sure you have libdjl_torch.so file under your app lib folder. For example:

[edgargao]

@xyang16
Hi,Yang.I had add this static block in my code,func ModelZoo.loadModel(criteria) succesed. But, it throw an new exception,like this: ai.djl.engine.EngineException: No deep learning engine found. Please refer to https://github.com/deepjavalibrary/djl/blob/master/docs/development/troubleshooting.md for more details. at ai.djl.engine.Engine.getInstance(Engine.java:131) at ai.djl.ndarray.NDManager.newBaseManager(NDManager.java:115)

The exception happened in this code : NDManager ndManager = NDManager.newBaseManager();

I has execute func Engine.debugEnvironment()， the log is: ----------- System Properties ----------- http.agent: Dalvik/2.1.0 (Linux; U; Android 8.1.0; LD_86x86 Build/OPM8.190605.003) user.home: DJL_CACHE_DIR: /sdcard/Files java.io.tmpdir: /data/user/0/com.exapmle.multi/cache --------- Environment Variables --------- SYSTEMSERVERCLASSPATH: /system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar:/system/framework/com.android.location.provider.jar PATH: /sbin:/system/sbin:/system/bin:/system/xbin:/vendor/bin:/vendor/xbin:/opt/ember-host/bin:/lgw/sbin ANDROID_DATA: /data ANDROID_SOCKET_zygote_secondary: 9 RK_ADEPT_DEVICE_TYPE: mobile RK_ADEPT_DEVICE_SALT_FILE: /mnt/sdcard/.adobe-digital-editions/devicesalt RK_ADOBE_DE_MOBILE: 1 ASEC_MOUNTPOINT: /mnt/asec ADOBE_FONTS_DIR: /system/fonts/adobefonts/ RK_ADEPT_ACTIVATION_FILE: /mnt/sdcard/.adobe-digital-editions/activation.xml EXTERNAL_STORAGE: /sdcard ANDROID_BOOTLOGO: 1 ANDROID_ASSETS: /system/app EBOOK_PAGE_VISIBLE_NUMBER: 2　 ANDROID_STORAGE: /storage RK_ADOBE_DE_DOC_FOLDER: /mnt/sdcard/Digital Editions ANDROID_ROOT: /system DOWNLOAD_CACHE: /data/cache RK_ADEPT_DEVICE_FILE: /mnt/sdcard/.adobe-digital-editions/device.xml BOOTCLASSPATH: /system/framework/core-oj.jar:/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/bouncycastle.jar:/system/framework/apache-xml.jar:/system/framework/legacy-test.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/org.apache.http.legacy.boot.jar:/system/framework/android.hidl.base-V1.0-java.jar:/system/framework/android.hidl.manager-V1.0-java.jar -------------- Directories -------------- temp directory: /data/user/0/com.exapmle.multi/cache DJL cache directory: /sdcard/Files Engine cache directory: /sdcard/Files ------------------ CUDA ----------------- GPU Count: 0 ----------------- Engines --------------- DJL version: 0.0 Default Engine: null

[frankfliu]

@edgargao Engine class failed to initialize because of ServiceLoader doesn't work as system app. So it cannot find Engines in the class path, and default engine is always null.

NDManager.newBaseManager() call uses default engine which will also fail.

As a workaround you can create NDManager with:

    NDManager.newBaseManager(Device.cpu(), "PyTorch");

[edgargao]

@frankfliu
Thanks,my project can right worked.

[edgargao]

@frankfliu
I want to load model from assets dir.How can i doing?

[frankfliu]

You can load pytorch model from InputStream, however this is not compatible with Criteria API, you have to manually load model and translator

        Translator<I, O> translator = new MyTranslator();
        Model model = Model.newInstance("mode", "PyTorch");
        InputStream is = AssetManager.open("mymodel.pt");
        model.load(is);
        ZooModel<I, O> zooModel = new ZooModel<>(model, translator);