“Failed to register op package” Error when running./demo_qwen_npu createContext()

[HirokenOvo]

Problem Description

An error occurred when running ./demo_qwen_npu.

Here is the content of my run.sh：

• The directory structure for adb push is:

temp_push_folder/
└── mllm
    ├── bin
    │   ├── demo_qwen_npu
    │   └── main_qwen_npu
    ├── models
    │   ├── qwen-1.5-1.8b-chat-int8.mllm
    │   └── qwen-1.5-1.8b-chat-q4k.mllm
    ├── qnn-lib
    │   ├── libQnnHtpOptraceProfilingReader.so
    │   ├── libQnnHtpPrepare.so
    │   ├── libQnnHtpProfilingReader.so
    │   ├── libQnnHtp.so
    │   ├── libQnnHtpV75CalculatorStub.so
    │   ├── libQnnHtpV75Skel.so
    │   ├── libQnnHtpV75Stub.so
    │   ├── libQnnLLaMAPackage_CPU.so
    │   └── libQnnLLaMAPackage_HTP.so
    └── vocab
        ├── qwen_merges.txt
        └── qwen_vocab.mllm

• The execution command is adb shell "cd /data/local/tmp/mllm/bin && export LD_LIBRARY_PATH=/vendor/lib64:/data/local/tmp/mllm/qnn-lib && export ADSP_LIBRARY_PATH=\"/vendor/lib64:/data/local/tmp/mllm/qnn-lib\" && export DSP_LIBRARY_PATH=\"/vendor/lib64\" && ./demo_qwen_npu".

By checking the source code call chain, the problem was found during the warm-up stage. The specific call path is as follows:

1. On line 46 of the examples/demo_qwen_npu.cpp file, the model.generate() function is called.
2. This call further triggers the QNNBackend::onSetUpStart() function in the src/backends/qnn/QNNBackend.cpp file.
3. In the QNNBackend::onSetUpStart() function, the this->createContext() function is called on line 218.
4. Then, when the m_qnnFunctionPointers.qnnInterface.contextCreate() function is called on line 642 of the createContext() function (located in the src/backends/qnn/QNNBackend.cpp file), an error occurs.

The error code indicates that the operator registration failed, rather than the context creation failure. However, judging from the INFO information, the custom operator seems to have been successfully registered.

The minimal reproduction code is as follows:

#include "QNNBackend.hpp"
#include "QNNMemoryManager.hpp"
#include <vector>
#include <memory>
#include <iostream>

using namespace mllm;

int main() {
    QNNBackend *qnnbackend =static_cast<QNNBackend*>(GetBackendCreator(MLLM_QNN)->create({}));
    if (StatusCode::SUCCESS != qnnbackend->createContext()) {
        std::cout << "Context Creation failure" << std::endl;
        return -1;
    }
    std::cout << "Test Passed" << std::endl;
}

Environment Information

1. QNN Version: V2.21.0.240401
2. Hexagon SDK Version: V5.5.3.0
3. Android NDK Version: android-ndk-r26c
4. Device Used: Lenovo Legion Y700 Tablet (Processor: 8gen3, RAM: 12GB, Memory Expansion: 9GB)

This problem has troubled me for a long time, I hope to get suggestions and solutions to this problem. Thank you very much!

[yangyyj]

请问您解决这个问题了吗

[HirokenOvo]

请问您解决这个问题了吗

Unfortunately, I haven't been able to solve it yet

[chenghuaWang]

cc @oreomaker @liang1232018

[hb-jw]

请问您解决这个问题了吗？

[HirokenOvo]

请问您解决这个问题了吗？

No, and I’ve stopped using this project. It seems like many people have encountered this same issue, so you could try asking others in this issue or contributors.

[oreomaker]

It seems that DSP_LIBRARY_PATH should not use as it will specify the QNN lib in system, which is not accessible for developers. The QNN library in qnn-lib is all needed to run the program.