LlaVA Model Loads Sucessfully Failing inference app crashes after image i/p adding logs for Reference

[Vinaysukhesh98]

🐛 Describe the bug

Application Crashes after i/p of image on MI15 device find logs below :

Discord message link : https://discord.com/channels/1334270993966825602/1336777807509979188/1349588357612568577

025-03-13 21:36:25.555  4077-4077  HandWritingStubImpl     com.example.executorchllamademo      I  refreshLastKeyboardType: 1
2025-03-13 21:36:25.555  4077-4077  HandWritingStubImpl     com.example.executorchllamademo      I  getCurrentKeyboardType: 1
2025-03-13 21:36:25.557 18376-18376 BaseInputMethodService  com.sohu.inputmethod.sogou.xiaomi    E  onStartInput app:com.example.executorchllamademo restarting:false
2025-03-13 21:36:25.564  4077-10239 ETLogging               com.example.executorchllamademo      D  Corrected RGB pixels (first 10 pixels): R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF R=0xFF G=0xFF B=0xFF 
2025-03-13 21:36:25.564  4077-10239 ETLogging               com.example.executorchllamademo      D  Llava start prefill image
2025-03-13 21:36:25.638 16878-17580 ActivityManagerWrapper  com.miui.home                        E  getRecentTasksForceIncludingTaskIdIfValid: getRecentTasks: size=1
                                                                                                     mainTaskId=66   userId=0   windowMode=1   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.executorchllamademo/com.example.executorchllamademo.MainActivity} }
2025-03-13 21:36:26.094  4077-10239 scudo                   com.example.executorchllamademo      I  Scudo ERROR: CHECK failed @ external/scudo/standalone/list.h:215 (Prev->Next) == (X) ((u64)op1=18410715280977166336, (u64)op2=72058109513912256)
2025-03-13 21:36:26.095  4077-10239 libc                    com.example.executorchllamademo      I  handling signal: 6
2025-03-13 21:36:26.095  4077-10239 libc                    com.example.executorchllamademo      A  Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 10239 (pool-4-thread-1), pid 4077 (utorchllamademo)
2025-03-13 21:36:26.095  4077-10239 libc                    com.example.executorchllamademo      I  debuggerd_dispatch_pseudothread start. crashing tid: 10239
2025-03-13 21:36:26.157  4077-10239 libc                    com.example.executorchllamademo      I  crash_dump pid: 10245
2025-03-13 21:36:26.549 10247-10247 DEBUG                   crash_dump64                         A  Cmdline: com.example.executorchllamademo
2025-03-13 21:36:26.549 10247-10247 DEBUG                   crash_dump64                         A  pid: 4077, tid: 10239, name: pool-4-thread-1  >>> com.example.executorchllamademo <<<
2025-03-13 21:36:26.549 10247-10247 DEBUG                   crash_dump64                         A        #08 pc 0000000000e89c5c  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.549 10247-10247 DEBUG                   crash_dump64                         A        #09 pc 0000000000e83ba0  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (torch::executor::native::custom_sdpa_out(executorch::runtime::KernelRuntimeContext&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, long, std::__ndk1::optional<executorch::runtime::etensor::Tensor> const&, double, bool, std::__ndk1::optional<double>, executorch::runtime::etensor::Tensor&)+772) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #10 pc 0000000000e84170  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (torch::executor::native::sdpa_with_kv_cache_out(executorch::runtime::KernelRuntimeContext&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor&, executorch::runtime::etensor::Tensor&, long, long, std::__ndk1::optional<executorch::runtime::etensor::Tensor> const&, double, bool, std::__ndk1::optional<double>, executorch::runtime::etensor::Tensor&)+832) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #11 pc 0000000000e8ec90  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (void executorch::extension::kernel_util_internal::call_functor_with_args_from_stack<executorch::extension::kernel_util_internal::CompileTimeFunctionPointer<executorch::runtime::etensor::Tensor& (executorch::runtime::KernelRuntimeContext&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor&, executorch::runtime::etensor::Tensor&, long, long, std::__ndk1::optional<executorch::runtime::etensor::Tensor> const&, double, bool, std::__ndk1::optional<double>, executorch::runtime::etensor::Tensor&), &torch::executor::native::sdpa_with_kv_cache_out(executorch::runtime::KernelRuntimeContext&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor&, executorch::runtime::etensor::Tensor&, long, long, std::__ndk1::optional<executorch::runtime::etensor::Tensor> const&, double, bool, std::__ndk1::optional<double>, executorch::runtime::etensor::Tensor&)>, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul, 8ul, 9ul, 10ul, 11ul, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor&, executorch::runtime::etensor::Tensor&, long, long, std::__ndk1::optional<executorch::runtime::etensor::Tensor> const&, double, bool, std::__ndk1::optional<double>, executorch::runtime::etensor::Tensor&>(executorch::runtime::KernelRuntimeContext&, executorch::runtime::EValue**, std::__ndk1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul, 7ul, 8ul, 9ul, 10ul, 11ul>, executorch::extension::kernel_util_internal::typelist<executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor const&, executorch::runtime::etensor::Tensor&, executorch::runtime::etensor::Tensor&, long, long, std::__ndk1::optional<executorch::runtime::etensor::Tensor> const&, double, bool, std::__ndk1::optional<double>, executorch::runtime::etensor::Tensor&>*)+300) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #12 pc 0000000000ee8494  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (executorch::runtime::Method::execute_instruction()+636) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #13 pc 0000000000ee8d30  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (executorch::runtime::Method::execute()+132) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #14 pc 0000000000eef78c  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (executorch::extension::Module::execute(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>> const&, std::__ndk1::vector<executorch::runtime::EValue, std::__ndk1::allocator<executorch::runtime::EValue>> const&)+360) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #15 pc 0000000000eb5d04  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (example::LlavaImagePrefiller::prefill(executorch::extension::llm::Image&, long&)+752) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #16 pc 0000000000eb3a60  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (example::LlavaRunner::prefill_images(std::__ndk1::vector<executorch::extension::llm::Image, std::__ndk1::allocator<executorch::extension::llm::Image>>&, long&)+80) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #17 pc 00000000001f0ebc  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (executorch_jni::ExecuTorchLlamaJni::prefill_images(facebook::jni::alias_ref<_jintArray*>, int, int, int, long)+788) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #18 pc 00000000001f1308  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (facebook::jni::detail::MethodWrapper<facebook::jni::basic_strong_ref<_jlongArray*, facebook::jni::LocalReferenceAllocator> (executorch_jni::ExecuTorchLlamaJni::*)(facebook::jni::alias_ref<_jintArray*>, int, int, int, long), &executorch_jni::ExecuTorchLlamaJni::prefill_images(facebook::jni::alias_ref<_jintArray*>, int, int, int, long), executorch_jni::ExecuTorchLlamaJni, facebook::jni::basic_strong_ref<_jlongArray*, facebook::jni::LocalReferenceAllocator>, facebook::jni::alias_ref<_jintArray*>, int, int, int, long>::dispatch(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, long&&)+124) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #19 pc 00000000001f11ec  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/lib/arm64/libexecutorch.so (facebook::jni::detail::FunctionWrapper<facebook::jni::basic_strong_ref<_jlongArray*, facebook::jni::LocalReferenceAllocator> (*)(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, long&&), facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*, facebook::jni::basic_strong_ref<_jlongArray*, facebook::jni::LocalReferenceAllocator>, facebook::jni::alias_ref<_jintArray*>, int, int, int, long>::call(_JNIEnv*, _jobject*, _jintArray*, int, int, int, long, facebook::jni::basic_strong_ref<_jlongArray*, facebook::jni::LocalReferenceAllocator> (*)(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, long&&))+124) (BuildId: 4246dcf4bbc396860c52fa4d9aa58ec3dc7caa92)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #26 pc 000000000036346c  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/base.apk (org.pytorch.executorch.LlamaModule.prefillImages+0)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #32 pc 0000000000006ff0  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/base.apk (com.example.executorchllamademo.MainActivity.lambda$showMediaPreview$19$com-example-executorchllamademo-MainActivity+0)
2025-03-13 21:36:26.550 10247-10247 DEBUG                   crash_dump64                         A        #38 pc 0000000000005be4  /data/app/~~Ng5TqUGJINyqaYKR9bko6g==/com.example.executorchllamademo-bkZel8xguZ5dm2TaVLrcIg==/base.apk (com.example.executorchllamademo.MainActivity$$ExternalSyntheticLambda10.run+0)
2025-03-13 21:36:26.588 16878-17580 ActivityManagerWrapper  com.miui.home                        E  getRecentTasksForceIncludingTaskIdIfValid: getRecentTasks: size=1
                                                                                                     mainTaskId=66   userId=0   windowMode=1   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.executorchllamademo/com.example.executorchllamademo.MainActivity} }
2025-03-13 21:36:26.625 16878-17580 ActivityManagerWrapper  com.miui.home                        E  getRecentTasksForceIncludingTaskIdIfValid: getRecentTasks: size=1
                                                                                                     mainTaskId=66   userId=0   windowMode=1   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.executorchllamademo/com.example.executorchllamademo.MainActivity} }
2025-03-13 21:36:26.692 16878-17580 ActivityManagerWrapper  com.miui.home                        E  getRecentTasksForceIncludingTaskIdIfValid: getRecentTasks: size=1
                                                                                                     mainTaskId=66   userId=0   windowMode=1   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.executorchllamademo/com.example.executorchllamademo.MainActivity} }
2025-03-13 21:36:26.696 16878-17580 ActivityManagerWrapper  com.miui.home                        E  getRecentTasksForceIncludingTaskIdIfValid: getRecentTasks: size=1
                                                                                                     mainTaskId=66   userId=0   windowMode=1   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.executorchllamademo/com.example.executorchllamademo.MainActivity} }
---------------------------- PROCESS ENDED (4077) for package com.example.executorchllamademo ----------------------------
2025-03-13 21:36:27.536 16878-17580 ActivityManagerWrapper  com.miui.home                        E  getRecentTasksForceIncludingTaskIdIfValid: getRecentTasks: size=1
                                                                                                     mainTaskId=66   userId=0   windowMode=1   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.executorchllamademo/com.example.executorchllamademo.MainActivity} }
2025-03-13 21:36:27.537 16878-17580 ActivityManagerWrapper  com.miui.home                        E  getRecentTasksForceIncludingTaskIdIfValid: getRecentTasks: size=1
                                                                                                     mainTaskId=66   userId=0   windowMode=1   baseIntent=Intent { act=android.intent.action.MAIN flag=268435456 cmp=ComponentInfo{com.example.executorchllamademo/com.example.executorchllamademo.MainActivity} }

Versions

Application Crashes after i/p of image on MI15 device

[kirklandsign]

Hi @Vinaysukhesh98 does this work with LLAVA runner binary? https://github.com/pytorch/executorch/blob/main/examples/models/llava/runner/CMakeLists.txt#L41

[Vinaysukhesh98]

Hi @Vinaysukhesh98 does this work with LLAVA runner binary? https://github.com/pytorch/executorch/blob/main/examples/models/llava/runner/CMakeLists.txt#L41
not it is not running

./cmake-out/examples/models/llava/llava_main -model_path '/home//Documents/llava.pte' -tokenizer_path '/home/mbuhyd/Documents/llava_tokenizer.bin' -image_path '/home//Documents/llavademo2.jpg' -prompt "Describe the image"
I 00:00:00.000392 executorch:cpuinfo_utils.cpp:62] Reading file /sys/devices/soc0/image_version
I 00:00:00.000418 executorch:cpuinfo_utils.cpp:78] Failed to open midr file /sys/devices/soc0/image_version
I 00:00:00.000438 executorch:cpuinfo_utils.cpp:91] Reading file /sys/devices/system/cpu/cpu0/regs/identification/midr_el1
I 00:00:00.000446 executorch:cpuinfo_utils.cpp:100] Failed to open midr file /sys/devices/system/cpu/cpu0/regs/identification/midr_el1
I 00:00:00.000448 executorch:cpuinfo_utils.cpp:116] CPU info and manual query on # of cpus dont match.
I 00:00:00.000450 executorch:main.cpp:77] Resetting threadpool with num threads = 0
I 00:00:00.000455 executorch:multimodal_runner.h:45] Creating Multimodal LLM runner: model_path=/home/mbuhyd/Documents/llava.pte, tokenizer_path=/home/mbuhyd/Documents/llava_tokenizer.bin
terminate called after throwing an instance of 'c10::Error'
what(): PytorchStreamReader failed reading zip archive: failed finding central directory
Exception raised from valid at /pytorch/caffe2/serialize/inline_container.cc:239 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x98 (0x7d2859d8b218 in /home/mbuhyd/anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0xe0 (0x7d2859d1ff72 in /home/mbuhyd/anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libc10.so)
frame #2: caffe2::serialize::PyTorchStreamReader::valid(char const*, char const*) + 0x340 (0x7d285e3e2100 in /home/mbuhyd/anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
frame #3: caffe2::serialize::PyTorchStreamReader::init() + 0xac (0x7d285e3e449c in /home/mbuhyd/anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
frame #4: caffe2::serialize::PyTorchStreamReader::PyTorchStreamReader(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0x156 (0x7d285e3e8056 in /home/mbuhyd/anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
frame #5: torch::jit::import_ir_module(std::shared_ptrtorch::jit::CompilationUnit, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::optionalc10::Device, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > >&, bool, bool) + 0x2b1 (0x7d285f5699f1 in /home//anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
frame #6: torch::jit::import_ir_module(std::shared_ptrtorch::jit::CompilationUnit, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::optionalc10::Device, bool) + 0x9c (0x7d285f569d1c in /home//anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
frame #7: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::optionalc10::Device, bool) + 0xc4 (0x7d285f569e24 in /home//anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
frame #8: torch::serialize::InputArchive::load_from(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::optionalc10::Device) + 0x39 (0x7d285fc34789 in /home/mbuhyd/anaconda3/envs/llava-March12/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
frame #9: + 0x3816c (0x614ca966116c in ./cmake-out/examples/models/llava/llava_main)
frame #10: + 0x1f261 (0x614ca9648261 in ./cmake-out/examples/models/llava/llava_main)
frame #11: + 0x29d90 (0x7d2859629d90 in /lib/x86_64-linux-gnu/libc.so.6)
frame #12: __libc_start_main + 0x80 (0x7d2859629e40 in /lib/x86_64-linux-gnu/libc.so.6)
frame #13: + 0x20795 (0x614ca9649795 in ./cmake-out/examples/models/llava/llava_main)

Aborted