iOS app crashes on start if ARCore Extensions is installed

[spkoreaa]

Plugin Version or Commit ID

v0.16.1

Unity Version

6000.0.29f1

Your Host OS

macOS Ventura 13.7.1

Target Platform

iOS

[Windows Only] Visual Studio C++ and Windows SDK Version

No response

[Linux Only] GCC/G++ and GLIBC Version

No response

[Android Only] Android Build Tools and NDK Version

No response

[iOS Only] XCode Version

iOS 18.3.2

Command Sequences

bt all

Log

objc[2029]: Class KalmanFilter is implemented in both /System/Library/PrivateFrameworks/CinematicFraming.framework/CinematicFraming (0x1fcc75510) and /private/var/containers/Bundle/Application/DE268BA9-4030-4729-BB95-B98D634EB143/ARCoreExtention.app/Frameworks/MediaPipeUnity.framework/MediaPipeUnity (0x10943ba60). One of the two will be used. Which one is undefined.
(lldb) bt all* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x6013c8148)    frame #0: 0x0000000108f7f680 MediaPipeUnity___lldb_unnamed_symbol57892 + 140    frame #1: 0x0000000108f7f308 MediaPipeUnitymediapipe::internal::GpuBufferStorageRegistry::Register(std::__1::function<std::__1::shared_ptrmediapipe::internal::GpuBufferStorage (int, int, mediapipe::GpuBufferFormat)>, std::__1::vector<mediapipe::TypeId, std::__1::allocatormediapipe::TypeId>) + 300    frame #2: 0x0000000107e0a7ec MediaPipeUnity___lldb_unnamed_symbol24787 + 168    frame #3: 0x0000000107e143a4 MediaPipeUnity___lldb_unnamed_symbol25015 + 132    frame #4: 0x00000001b9ab07a8 dyldinvocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 592    frame #5: 0x00000001b9b021ac dyldinvocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 172    frame #6: 0x00000001b9ab1478 dyldinvocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 496    frame #7: 0x00000001b9ab120c dylddyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 300    frame #8: 0x00000001b9ab0fc0 dylddyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 192    frame #9: 0x00000001b9aa8000 dylddyld3::MachOFile::forEachInitializerPointerSection(Diagnostics&, void (unsigned int, unsigned int, bool&) block_pointer) const + 160    frame #10: 0x00000001b9aa7e18 dylddyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 432    frame #11: 0x00000001b9aa7954 dylddyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 176    frame #12: 0x00000001b9aa87bc dylddyld4::JustInTimeLoader::runInitializers(dyld4::RuntimeState&) const + 36    frame #13: 0x00000001b9aa6b48 dylddyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&, dyld3::Array<dyld4::Loader const*>&) const + 312    frame #14: 0x00000001b9aa6ae8 dylddyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&, dyld3::Array<dyld4::Loader const*>&) const + 216    frame #15: 0x00000001b9abd114 dylddyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_0::operator()() const + 180    frame #16: 0x00000001b9abce60 dylddyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 412    frame #17: 0x00000001b9aba0a8 dylddyld4::APIs::dlopen_from(char const*, int, void*) + 2432    frame #18: 0x00000001938ddd88 CoreFoundation_CFBundleDlfcnLoadFramework + 164    frame #19: 0x000000019388a988 CoreFoundation_CFBundleLoadExecutableAndReturnError + 476    frame #20: 0x0000000192461094 Foundation-[NSBundle loadAndReturnError:] + 212  * frame #21: 0x00000001048f008c ARCoreExtentionUnityFrameworkLoad() at main.mm:10:37 [opt]    frame #22: 0x00000001048f0114 ARCoreExtentionmain(argc=1, argv=0x000000016b513800) at main.mm:25:18 [opt]    frame #23: 0x00000001b9acdde8 dyldstart + 2724

Additional Context

I am using MediaPipeUnityPlugin-all-stripped.zip v0.16.1 to implement the hand tracking into my AR app. I am using Unity ARFoundation + ARCore Extensions with ARCloud Anchors . I have successfully connected the ARCamera as an input to the hand tracking and everything works well as expected on Android with no issues. However, when I try to build it for iOS EXC_BAD_ACCESS on UnityFrameLoad(), which I can tell MediaPipeUnity issues with registering gpubuffer. I tried to back-trace the thread using "bt all" so you can see the whole error log. The problem is getting invoked when I specifically set the Project Settings -> ARCore Extensions -> iOS Support Enabled to True and build the project. Without iOS Supoort Enabled everything works fine with no issue. Removing either of them (MediaPipeUnity or ARCore Extensions iOS enabled) is eliminating the problem, however I should use both.

[homuler]

Please share the code to reproduce the error and tell the exact line where it crashes.

[homuler]

Also, I'd like you to use MediaPipeUnityPlugin-all.zip instead of MediaPipeUnityPlugin-all-stripped.zip.
The symbols are missing, so there's hardly any information in the stack trace.

[spkoreaa]

Also, I'd like you to use MediaPipeUnityPlugin-all.zip instead of MediaPipeUnityPlugin-all-stripped.zip. The symbols are missing, so there's hardly any information in the stack trace.

Hello, sorry for late reply. Yeah, as you suggested I have tested it with all available releases, including MediaPipeUnityPlugin-all.zip but the issue is the same, the error log also the same. Which part of the code you exactly need? Or do you need the whole project? And the app is crashing way before, when it is loading the Frameworks. You can check the log I have provided before. For the hand tracking scripts, I am using these two custom scripts (I have attached it below). Also I have attached a screenshot from Xcode error on build. The build is successful, the error arises when app starts and initializes. If you need whole project I can share it with you

HandGestureTracker.txt
HandLandmarkDetector.txt

[spkoreaa]

When I checked the Issues section on github I saw the similar one, issue number: #544 EXC_BAD_ACCESS on iOS build (CreateCVPixelBufferForImageFramePacket crash). I have tried the solutions there, but no success, I think it is different situation from mine

[homuler]

the error log also the same.

Actually it is not the same, is it?

frame # 2: 0x0000000107e0a7ec MediaPipeUnity___lldb_unnamed_symbol24787 + 168

In the stack trace shown in the screenshot, the symbol name is properly displayed instead of just unnamed symbol, right?

To be honest, even with this stack trace, it's hard to determine the exact cause. I can guess which part of the process it's crashing in, but I can't pinpoint the exact location.
We might get more insight if you build the library in debug mode (bazel build -c dbg).

Of course, I can't fully debug your application under free support, but if you can provide a minimal setup that I can easily reproduce on my end, I think I can look into it a bit more.

[spkoreaa]

the error log also the same.

Actually it is not the same, is it?

frame # 2: 0x0000000107e0a7ec MediaPipeUnity___lldb_unnamed_symbol24787 + 168

In the stack trace shown in the screenshot, the symbol name is properly displayed instead of just unnamed symbol, right?

To be honest, even with this stack trace, it's hard to determine the exact cause. I can guess which part of the process it's crashing in, but I can't pinpoint the exact location. We might get more insight if you build the library in debug mode (bazel build -c dbg).

Of course, I can't fully debug your application under free support, but if you can provide a minimal setup that I can easily reproduce on my end, I think I can look into it a bit more.

Thank you so much, for using your valuable time~ Like you asked, I made minimal setup in Unity project so that you can reproduce the same issue. I have uploaded it to the Google drive (Unity project), if you can let me know your email address I would like to share it with you please.

[homuler]

If it's really just the minimal setup and doesn't include any confidential code, would it be possible to share it as is?

[homuler]

@spkoreaa Since the API key was visible in the screenshot, I deleted the entire comment. Just to be safe, I recommend regenerating the API key.

[homuler]

Could you try reducing the amount of unnecessary code a bit more?
The project you shared seems to contain a lot of code that isn’t needed to reproduce the issue.

Based on your current hypothesis, it sounds like a package conflict might be the cause.
If that's the case, I’d appreciate it if you could reproduce the issue in a minimal, empty project that includes only the relevant packages (plus the minimum code needed to load the libraries), and then share that code instead.

In other words, I’d like you to prove that the issue is not caused by the code you wrote.

[spkoreaa]

Could you try reducing the amount of unnecessary code a bit more? The project you shared seems to contain a lot of code that isn’t needed to reproduce the issue.

Based on your current hypothesis, it sounds like a package conflict might be the cause. If that's the case, I’d appreciate it if you could reproduce the issue in a minimal, empty project that includes only the relevant packages (plus the minimum code needed to load the libraries), and then share that code instead.

In other words, I’d like you to prove that the issue is not caused by the code you wrote.

Oh no worries, I didn't share the real API key, so it is okay. Like you asked, just in case I have created two minimal project. I haven't included any custom code, or any code written by me. The project only includes the MediaPipeUnity release package, Unity ARFoundation(for ARCore to work), ARCore Extensions. There is nothing else is included. This project includes https://drive.google.com/drive/folders/1SlH0LfhQUrxJqNc985pkQ-VjnYhm8kDH?usp=sharing MediaPipeUnityPlugin-all.zip. This one https://drive.google.com/drive/folders/1S3l6h05-HG_-GG9ccshL2AGdoVnqLloU?usp=sharing includes MediaPipeUnity.0.16.1.unitypackage. For both I used latest MediaPipeUnity v0.16.1. You can choose either of them, just if you think which one is more minimal for you to test. Like you requested both projects made with minimal assets on empty project with only your MediaPipeUnity plugin, ARFoundation (ARCore + ARKit), Google ARCore Extensions.

[spkoreaa]

@homuler Please note that even though either ARCore Extensions or MediaPipeUnity are not initialized on active scene the issue is same. My suspect is maybe Script Compilation -> Scripting Define Symbols, or Macros related to package conflicts. My another suspect is IOSResolver or CocoaPods, because the issue arises only on iOS build, Android works perfectly

[spkoreaa]

@homuler Any progress so far?

[homuler]

I have confirmed that the issue can be reproduced by following the steps below:

1. Create an empty project
2. Import AR Foundation, ARCore Extensions (https://github.com/google-ar/arcore-unity-extensions.git#arf6), and MediaPipeUnityPlugin (com.github.homuler.mediapipe-0.16.1.tgz)
3. Enable iOS support for ARCore Extensions
4. Build and run