Segmentation Fault when implementing llama/stories110M Android phone deployment

[BESTTOOLBOX]

🐛 Describe the bug

I encountered a segmentation fault when implementing llama/stories110M Android phone deployment according to the #https://github.com/pytorch/executorch/blob/main/examples/models/llama2/README.md. I put xnnpack_stories110M.pte, tokenizer.bin and llama_main compiled with xnn backend into the same directory and executed llama_main. The following is the error message: Here is the logcat output: The following is the information parsed by addr2line. It seems that there was an error in the memory allocation. #00 pc 0000000002ff1afc /data/local/tmp/jiaxgeng/stories/llama_main (xnn_pack_qs8_qc4w_gemm_bl_goi_w_nr8_kr4+2696) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/third-party/XNNPACK/src/packing.c:548 07-12 10:11:23.542 11569 11569 F DEBUG : #01 pc 0000000002ff3604 /data/local/tmp/jiaxgeng/stories/llama_main (xnn_pack_qs8_qc4w_gemm_bl_goi_w+1512) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/third-party/XNNPACK/src/packing.c:753 07-12 10:11:23.542 11569 11569 F DEBUG : #02 pc 00000000030e79bc /data/local/tmp/jiaxgeng/stories/llama_main (xnn_create_fully_connected_nc_qd8_f32_qb4w+3240) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/third-party/XNNPACK/src/operators/fully-connected-nc.c:737 07-12 10:11:23.542 11569 11569 F DEBUG : #03 pc 00000000030e1acc /data/local/tmp/jiaxgeng/stories/llama_main (create_fully_connected_operator+3156) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/third-party/XNNPACK/src/subgraph/fully-connected.c:237 07-12 10:11:23.542 11569 11569 F DEBUG : #04 pc 0000000002d4791c /data/local/tmp/jiaxgeng/stories/llama_main (xnn_create_runtime_v4+1644) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/third-party/XNNPACK/src/runtime.c:575 07-12 10:11:23.542 11569 11569 F DEBUG : #05 pc 0000000002d47260 /data/local/tmp/jiaxgeng/stories/llama_main (xnn_create_runtime_v3+104) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/third-party/XNNPACK/src/runtime.c:208 07-12 10:11:23.542 11569 11569 F DEBUG : #06 pc 0000000002d471e8 /data/local/tmp/jiaxgeng/stories/llama_main (xnn_create_runtime_v2+48) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/third-party/XNNPACK/src/runtime.c:193 07-12 10:11:23.542 11569 11569 F DEBUG : #07 pc 0000000000855d4c /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::xnnpack::delegate::XNNCompiler::compileModel(void const*, unsigned long, torch::executor::xnnpack::delegate::XNNExecutor*, torch::executor::MemoryAllocator*)+1760) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/runtime/XNNCompiler.cpp:1681 07-12 10:11:23.542 11569 11569 F DEBUG : #08 pc 0000000000861c88 /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::XnnpackBackend::init(torch::executor::BackendInitContext&, torch::executor::FreeableBuffer*, torch::executor::ArrayReftorch::executor::CompileSpec) const+184) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/backends/xnnpack/runtime/XNNPACKBackend.cpp:42 07-12 10:11:23.542 11569 11569 F DEBUG : #09 pc 000000000312bbd0 /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::BackendDelegate::Init(executorch_flatbuffer::BackendDelegate const&, torch::executor::Program const*, torch::executor::BackendInitContext&, torch::executor::BackendDelegate*)+992) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/runtime/executor/method.cpp:97 07-12 10:11:23.542 11569 11569 F DEBUG : #10 pc 000000000312a74c /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::Method::init(executorch_flatbuffer::ExecutionPlan*)+728) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/runtime/executor/method.cpp:596 07-12 10:11:23.542 11569 11569 F DEBUG : #11 pc 000000000312a298 /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::Method::load(executorch_flatbuffer::ExecutionPlan*, torch::executor::Program const*, torch::executor::MemoryManager*, torch::executor::EventTracer*)+84) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/runtime/executor/method.cpp:547 07-12 10:11:23.542 11569 11569 F DEBUG : #12 pc 00000000031345c4 /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::Program::load_method(char const*, torch::executor::MemoryManager*, torch::executor::EventTracer*) const+380) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/runtime/executor/program.cpp:246 07-12 10:11:23.542 11569 11569 F DEBUG : #13 pc 0000000003118424 /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::Module::load_method(std::__ndk1::basic_string<char, std::__ndk1::char_traits, std::__ndk1::allocator > const&)+888) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/extension/module/module.cpp:131 07-12 10:11:23.542 11569 11569 F DEBUG : #14 pc 000000000086cd78 /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::Runner::load()+312) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/examples/models/llama2/runner/runner.cpp:73 07-12 10:11:23.542 11569 11569 F DEBUG : #15 pc 000000000086fc94 /data/local/tmp/jiaxgeng/stories/llama_main (torch::executor::Runner::generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits, std::__ndk1::allocator > const&, int, std::__ndk1::function<void (std::__ndk1::basic_string<char, std::__ndk1::char_traits, std::__ndk1::allocator > const&)>, std::__ndk1::function<void (torch::executor::Runner::Stats const&)>)+236) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/examples/models/llama2/runner/runner.cpp:351 07-12 10:11:23.542 11569 11569 F DEBUG : #16 pc 0000000000863fe8 /data/local/tmp/jiaxgeng/stories/llama_main (main+480) (BuildId: 76fcd4977dfd60ce58d60c76d5f70bf61f77b471) /local/mnt/workspace/executorch/examples/models/llama2/main.cpp:75 07-12 10:11:23.542 11569 11569 F DEBUG : #17 pc 0000000000053e48 /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+108) (BuildId: 50118287324a156bc7be11d3d940c7be)

Here are my compile instructions for llama_main android. To see debug information, set type to debug. sudo cmake -DCMAKE_TOOLCHAIN_FILE=/local/mnt/workspace/android-ndk-r26d/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-33
-DCMAKE_INSTALL_PREFIX=cmake-out-android
-DCMAKE_BUILD_TYPE=Debug
-DEXECUTORCH_BUILD_EXTENSION_MODULE=ON
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON
-DEXECUTORCH_ENABLE_LOGGING=1
-DPYTHON_EXECUTABLE=python
-DEXECUTORCH_BUILD_XNNPACK=ON
-DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON
-DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON
-DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON
-Bcmake-out-android .

cmake --build cmake-out-android -j16 --target install --config Debug

sudo cmake -DCMAKE_TOOLCHAIN_FILE=/local/mnt/workspace/android-ndk-r26d/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-33
-DCMAKE_INSTALL_PREFIX=cmake-out-android
-DCMAKE_BUILD_TYPE=Debug
-DPYTHON_EXECUTABLE=python
-DEXECUTORCH_BUILD_XNNPACK=ON
-DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON
-DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON
-DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON
-Bcmake-out-android/examples/models/llama2
examples/models/llama2

sudo cmake --build cmake-out-android/examples/models/llama2 -j16 --config Debug

Versions

[pip3] executorch==0.4.0a0+8740c69 [pip3] numpy==2.0.0 [pip3] torch==2.5.0.dev20240618+cpu [pip3] torchao==0.1 [pip3] torchaudio==2.4.0.dev20240618+cpu [pip3] torchsr==1.0.4 [pip3] torchvision==0.20.0.dev20240618+cpu [conda] numpy 1.26.4 pypi_0 pypi [conda] torch 2.2.2 pypi_0 pypi [conda] torchaudio 2.2.2 pypi_0 pypi [conda] torchvision 0.17.2 pypi_0 pypi