cpp_client_telemetry icon indicating copy to clipboard operation
cpp_client_telemetry copied to clipboard

Crash in Microsoft::Applications::Events::TelemetrySystem::handleIncomingEventPrepared?

Open tachtevrenidis opened this issue 3 years ago • 17 comments

  • OneDsCppSdk.iOS (3.5.117.1)

Hello, we often see this crash in our AppCenter crashes. Is this a common/known issue? Is this something that is fixed in more recent versions of OneDS?

Steps to reproduce. We don't have a repro, just a stack trace.

What is the expected behavior? No crash?

What is the actual behavior? Crash

Additional context. Stack trace Screen Shot 2022-03-29 at 8 10 28 AM

tachtevrenidis avatar Mar 29 '22 15:03 tachtevrenidis

Crash while calling ILogConfiguration::operator[]. It would be good to have the repro, or at-least to know the scenario where crash is happening. If it is during Shutdown when LogManager::FlushAndTeardown is called, is it possible that application is accessing ILogConfiguration in some parallel thread. ILogConfiguration map is not thread safe, and shouldn't be called/used explicitly after LogManager initialization.

Also as mentioned above, good to have a C++ repro for the crash, else it is difficult to debug it. It's on iOS app, so in case @kindbe @eduardo-camacho have further comments to add.

lalitb avatar Mar 31 '22 18:03 lalitb

I had the same problem. But @lalitb i can't repro it. As you said, if it is caused by multi-threading, can it be changed to thread-safe? And the version i used is the latest v3.6.69, so @tachtevrenidis it's not fixed in new version.

Incident Identifier: 470597fe-450d-4d70-92bd-54f1ac57e8ea
CrashReporter Key:   38757214-F4B1-47D3-A4ED-32737E471AB5
Hardware Model:      iPhone11,6
Process:         XXX [648]
Path:            /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/XXX
Identifier:      XXX
Version:         22.4.400420001 (400420001)
Code Type:       arm64
Parent Process:   [1]

Date/Time:       2022-04-25T04:47:51.999Z
Launch Time:     2022-04-25T04:42:35Z
OS Version:      iPhone OS 14.7.1 (18G82)
Report Version:  104

Exception Type:  SIGSEGV
Exception Codes: SEGV_MAPERR at 0x283d6bbc80037
Crashed Thread:  2

Thread 2 Crashed:
0   XXX                        0x0000000100b0285c std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const (string:1459)
1   XXX                        0x0000000100b02be4 std::__1::__tree_node_base<void*>*& std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, CsProtocol::Value>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, CsProtocol::Value>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, CsProtocol::Value> > >::__find_equal<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(std::__1::__tree_end_node<std::__1::__tree_node_base<void*>*>*&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (map:537)
2   XXX                        0x0000000100b6a880 std::__1::pair<std::__1::__tree_iterator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Microsoft::Applications::Events::Variant>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Microsoft::Applications::Events::Variant>, void*>*, long>, bool> std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Microsoft::Applications::Events::Variant>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Microsoft::Applications::Events::Variant>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Microsoft::Applications::Events::Variant> > >::__emplace_unique_key_args<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::piecewise_construct_t const&, std::__1::tuple<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&>, std::__1::tuple<> >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::piecewise_construct_t const&&&, std::__1::tuple<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&>&&, std::__1::tuple<>&&) (__tree:2093)
3   XXX                        0x0000000100b269b8 Microsoft::Applications::Events::ILogConfiguration::operator[](char const*) (map:1560)
4   XXX                        0x0000000100b0f548 Microsoft::Applications::Events::RuntimeConfig_Default::IsClockSkewEnabled() (RuntimeConfig_Default.hpp:186)
5   XXX                        0x0000000100b43478 Microsoft::Applications::Events::OfflineStorageHandler::GetAndReserveRecords(std::__1::function<bool (Microsoft::Applications::Events::StorageRecord&&)> const&, unsigned int, Microsoft::Applications::Events::EventLatency, unsigned int) (OfflineStorageHandler.cpp:334)
6   XXX                        0x0000000100b36fd8 Microsoft::Applications::Events::StorageObserver::handleRetrieveEvents(std::__1::shared_ptr<Microsoft::Applications::Events::EventsUploadContext> const&) (StorageObserver.cpp:53)
7   XXX                        0x0000000100afacfc Microsoft::Applications::Events::TransmissionPolicyManager::uploadAsync(Microsoft::Applications::Events::EventLatency) (TransmissionPolicyManager.cpp:217)
8   XXX                        0x0000000100b483f8 Microsoft::Applications::Events::PlatformAbstraction::WorkerThread::threadFunc(void*) (WorkerThread.cpp:241)
9   XXX                        0x0000000100b48ae4 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (*)(void*), void*> >(void*) (type_traits:3918)
10  libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
11  libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 0:
0   MicrosoftCognitiveServicesSpeech     0x00000001044b5670 0x1040d4000 + 4068976
1   libsystem_c.dylib                    0x000000019bd165c4 exit + 24
2   UIKitCore                            0x000000019536b2f0 -[UIApplication _terminateWithStatus:] + 500
3   UIKitCore                            0x00000001949b78cc -[_UISceneLifecycleMultiplexer _evalTransitionToSettings:fromSettings:forceExit:withTransitionStore:] + 124
4   UIKitCore                            0x00000001949b74fc -[_UISceneLifecycleMultiplexer forceExitWithTransitionContext:scene:] + 216
5   UIKitCore                            0x0000000195360dc0 -[UIApplication workspaceShouldExit:withTransitionContext:] + 208
6   FrontBoardServices                   0x00000001a1f3c9a8 -[FBSUIApplicationWorkspaceShim workspaceShouldExit:withTransitionContext:] + 84
7   FrontBoardServices                   0x00000001a1f6b3d4 __63-[FBSWorkspaceScenesClient willTerminateWithTransitionContext:]_block_invoke_2 + 76
8   FrontBoardServices                   0x00000001a1f4f6a4 -[FBSWorkspace _calloutQueue_executeCalloutFromSource:withBlock:] + 236
9   FrontBoardServices                   0x00000001a1f6b36c __63-[FBSWorkspaceScenesClient willTerminateWithTransitionContext:]_block_invoke + 128
10  libdispatch.dylib                    0x000000019255b81c _dispatch_client_callout + 16
11  libdispatch.dylib                    0x000000019255f30c _dispatch_block_invoke_direct + 264
12  FrontBoardServices                   0x00000001a1f93fa0 __FBSSERIALQUEUE_IS_CALLING_OUT_TO_A_BLOCK__ + 44
13  FrontBoardServices                   0x00000001a1f93c30 -[FBSSerialQueue _targetQueue_performNextIfPossible] + 444
14  FrontBoardServices                   0x00000001a1f94184 -[FBSSerialQueue _performNextFromRunLoopSource] + 28
15  CoreFoundation                       0x00000001928e8990 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24
16  CoreFoundation                       0x00000001928e888c __CFRunLoopDoSource0 + 204
17  CoreFoundation                       0x00000001928e7b90 __CFRunLoopDoSources0 + 264
18  CoreFoundation                       0x00000001928e1b70 __CFRunLoopRun + 816
19  CoreFoundation                       0x00000001928e1308 CFRunLoopRunSpecific + 596
20  GraphicsServices                     0x00000001a9f64734 GSEventRunModal + 160
21  UIKitCore                            0x000000019535f75c -[UIApplication _run] + 1068
22  UIKitCore                            0x0000000195364fcc UIApplicationMain + 164
23  XXX                        0x0000000100801810 main (main.swift:17)
24  libdyld.dylib                        0x000000019259dcf8 start + 0

Thread 1:
0   libsystem_kernel.dylib               0x00000001c0b344fc mach_msg_trap + 8
1   CoreFoundation                       0x00000001928e7e58 __CFRunLoopServiceMachPort + 368
2   CoreFoundation                       0x00000001928e1cf8 __CFRunLoopRun + 1208
3   CoreFoundation                       0x00000001928e1308 CFRunLoopRunSpecific + 596
4   Foundation                           0x0000000193bd0d8c -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 228
5   Foundation                           0x0000000193bd0c58 -[NSRunLoop(NSRunLoop) runUntilDate:] + 88
6   UIKitCore                            0x0000000195414304 -[UIEventFetcher threadMain] + 512
7   Foundation                           0x0000000193d4230c __NSThread__start__ + 860
8   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
9   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 3:
0   libsystem_kernel.dylib               0x00000001c0b590cc __psynch_cvwait + 8
1   libc++.1.dylib                       0x00000001a74f3f98 std::__1::condition_variable::__do_timed_wait(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >) + 96
2   JavaScriptCore                       0x000000019cd61118 bmalloc::Scavenger::threadRunLoop() + 548
3   JavaScriptCore                       0x000000019cd60bcc bmalloc::Scavenger::threadEntryPoint(bmalloc::Scavenger*) + 12
4   JavaScriptCore                       0x000000019cd620e4 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (*)(bmalloc::Scavenger*), bmalloc::Scavenger*> >(void*) + 44
5   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
6   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 4:
0   XXX                        0x0000000100ccfc14 mach_exception_callback (PLCrashReporter.m:296)
1   XXX                        0x0000000100cdb88c exception_server_thread (PLCrashMachExceptionServer.m:780)
2   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
3   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 5:
0   libsystem_kernel.dylib               0x00000001c0b344fc mach_msg_trap + 8
1   CoreFoundation                       0x00000001928e7e58 __CFRunLoopServiceMachPort + 368
2   CoreFoundation                       0x00000001928e1cf8 __CFRunLoopRun + 1208
3   CoreFoundation                       0x00000001928e1308 CFRunLoopRunSpecific + 596
4   CoreFoundation                       0x00000001928e23f4 CFRunLoopRun + 60
5   XXX                        0x0000000100d22548 -[BCNRunLoopThread threadLoop] (BCNRunLoopThread.m:113)
6   Foundation                           0x0000000193d4230c __NSThread__start__ + 860
7   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
8   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 6:
0   libsystem_kernel.dylib               0x00000001c0b344fc mach_msg_trap + 8
1   CoreFoundation                       0x00000001928e7e58 __CFRunLoopServiceMachPort + 368
2   CoreFoundation                       0x00000001928e1cf8 __CFRunLoopRun + 1208
3   CoreFoundation                       0x00000001928e1308 CFRunLoopRunSpecific + 596
4   CoreFoundation                       0x00000001928e23f4 CFRunLoopRun + 60
5   CoreMotion                           0x000000019da78c78 CLClientCreateIso6709Notation + 196100
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 7:
0   libsystem_kernel.dylib               0x00000001c0b344fc mach_msg_trap + 8
1   CoreFoundation                       0x00000001928e7e58 __CFRunLoopServiceMachPort + 368
2   CoreFoundation                       0x00000001928e1cf8 __CFRunLoopRun + 1208
3   CoreFoundation                       0x00000001928e1308 CFRunLoopRunSpecific + 596
4   AudioSession                         0x000000019a390f20 GenericRunLoopThread::Entry(void*) + 160
5   AudioSession                         0x000000019a3930d8 CAPThread::Entry(CAPThread*) + 88
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 8:
0   libsystem_kernel.dylib               0x00000001c0b344fc mach_msg_trap + 8
1   CoreFoundation                       0x00000001928e7e58 __CFRunLoopServiceMachPort + 368
2   CoreFoundation                       0x00000001928e1cf8 __CFRunLoopRun + 1208
3   CoreFoundation                       0x00000001928e1308 CFRunLoopRunSpecific + 596
4   CFNetwork                            0x00000001931bfb38 _CFURLStorageSessionCopyCache + 64264
5   Foundation                           0x0000000193d4230c __NSThread__start__ + 860
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 9:
0   libsystem_kernel.dylib               0x00000001c0b344fc mach_msg_trap + 8
1   CoreFoundation                       0x00000001928e7e58 __CFRunLoopServiceMachPort + 368
2   CoreFoundation                       0x00000001928e1cf8 __CFRunLoopRun + 1208
3   CoreFoundation                       0x00000001928e1308 CFRunLoopRunSpecific + 596
4   WebCore                              0x00000001a00961ac RunWebThread(void*) + 684
5   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
6   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 10:
0   libsystem_kernel.dylib               0x00000001c0b5a0a4 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001de5e574c start_wqthread + 4

Thread 11:
0   libsystem_kernel.dylib               0x00000001c0b5a0a4 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001de5e574c start_wqthread + 4

Thread 12:
0   libsystem_kernel.dylib               0x00000001c0b5a0a4 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001de5e574c start_wqthread + 4

Thread 13:
0   libsystem_kernel.dylib               0x00000001c0b5a0a4 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001de5e574c start_wqthread + 4

Thread 14:
0   libsystem_kernel.dylib               0x00000001c0b5a0a4 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001de5e574c start_wqthread + 4

Thread 15:
0   libsystem_kernel.dylib               0x00000001c0b5a0a4 __workq_kernreturn + 8
1   libsystem_pthread.dylib              0x00000001de5e574c start_wqthread + 4

Thread 16:
0   libsystem_kernel.dylib               0x00000001c0b590cc __psynch_cvwait + 8
1   JavaScriptCore                       0x000000019cce7978 WTF::ParkingLot::parkConditionallyImpl(void const*, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 2776
2   JavaScriptCore                       0x000000019c522d1c bool WTF::Condition::waitUntil<WTF::Lock>(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 192
3   JavaScriptCore                       0x000000019ccb41e8 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 200
4   JavaScriptCore                       0x000000019cd0b84c WTF::Thread::entryPoint(WTF::Thread::NewThreadContext*) + 312
5   JavaScriptCore                       0x000000019cd0e00c WTF::wtfThreadEntryPoint(void*) + 12
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 17:
0   libsystem_kernel.dylib               0x00000001c0b590cc __psynch_cvwait + 8
1   JavaScriptCore                       0x000000019cce7978 WTF::ParkingLot::parkConditionallyImpl(void const*, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 2776
2   JavaScriptCore                       0x000000019c522d1c bool WTF::Condition::waitUntil<WTF::Lock>(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 192
3   JavaScriptCore                       0x000000019ccb41e8 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 200
4   JavaScriptCore                       0x000000019cd0b84c WTF::Thread::entryPoint(WTF::Thread::NewThreadContext*) + 312
5   JavaScriptCore                       0x000000019cd0e00c WTF::wtfThreadEntryPoint(void*) + 12
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 18:
0   libsystem_kernel.dylib               0x00000001c0b590cc __psynch_cvwait + 8
1   JavaScriptCore                       0x000000019cce7978 WTF::ParkingLot::parkConditionallyImpl(void const*, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 2776
2   JavaScriptCore                       0x000000019c522d1c bool WTF::Condition::waitUntil<WTF::Lock>(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 192
3   JavaScriptCore                       0x000000019ccb41e8 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 200
4   JavaScriptCore                       0x000000019cd0b84c WTF::Thread::entryPoint(WTF::Thread::NewThreadContext*) + 312
5   JavaScriptCore                       0x000000019cd0e00c WTF::wtfThreadEntryPoint(void*) + 12
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 19:
0   libsystem_kernel.dylib               0x00000001c0b590cc __psynch_cvwait + 8
1   JavaScriptCore                       0x000000019cce7978 WTF::ParkingLot::parkConditionallyImpl(void const*, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 2776
2   JavaScriptCore                       0x000000019c522d1c bool WTF::Condition::waitUntil<WTF::Lock>(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 192
3   JavaScriptCore                       0x000000019ccb41e8 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 200
4   JavaScriptCore                       0x000000019cd0b84c WTF::Thread::entryPoint(WTF::Thread::NewThreadContext*) + 312
5   JavaScriptCore                       0x000000019cd0e00c WTF::wtfThreadEntryPoint(void*) + 12
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 20:
0   libsystem_kernel.dylib               0x00000001c0b590cc __psynch_cvwait + 8
1   JavaScriptCore                       0x000000019cce7978 WTF::ParkingLot::parkConditionallyImpl(void const*, WTF::ScopedLambda<bool ()> const&, WTF::ScopedLambda<void ()> const&, WTF::TimeWithDynamicClockType const&) + 2776
2   JavaScriptCore                       0x000000019c522d1c bool WTF::Condition::waitUntil<WTF::Lock>(WTF::Lock&, WTF::TimeWithDynamicClockType const&) + 192
3   JavaScriptCore                       0x000000019ccb41e8 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() + 200
4   JavaScriptCore                       0x000000019cd0b84c WTF::Thread::entryPoint(WTF::Thread::NewThreadContext*) + 312
5   JavaScriptCore                       0x000000019cd0e00c WTF::wtfThreadEntryPoint(void*) + 12
6   libsystem_pthread.dylib              0x00000001de5dcbfc _pthread_start + 316
7   libsystem_pthread.dylib              0x00000001de5e5758 thread_start + 4

Thread 2 crashed with arm64 Thread State:
   x21: 0x000000016f9d2db8     x2: 0x0000000000000000    x16: 0x00000001de5d8a70     x3: 0xfffffffffffffff1
   x22: 0x000283d6bbc80020     x4: 0x0000000000000065    x17: 0x0000000000000002   cpsr: 0x0000000080000000
    x5: 0x0000000000000074    x23: 0x000283d6bbc80000     x6: 0x000000000000006d    x18: 0x0000000000000000
   x10: 0x0000000000000000     lr: 0x0000000100b02be4     x7: 0x000000016f9d2cc0    x24: 0x000000016f9d2e78
   x11: 0x0000000000000000     x8: 0x000283d6bbc80000    x19: 0x000000016f9d2d78    x25: 0x00000000000001bf
    x9: 0x0000000000000000    x12: 0x0000010000000000     fp: 0x000000016f9d2d10    x26: 0x00000000016926bc
   x13: 0x0000000000000000     pc: 0x0000000100b0285c    x27: 0x0000000282848d80    x14: 0x0000000000000000
   x20: 0x0000000283d7000e     x0: 0x000000016f9d2db8     sp: 0x000000016f9d2d00    x28: 0xfffffffffffffffa
   x15: 0x0000000000000000     x1: 0x000283d6bbc80020

Binary Images:
0x00000001007fc000 -        0x0000000103623fff +XXX arm64  <c189a7ec0d013a6e91e8356f25557e17> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/XXX
0x0000000103e0c000 -        0x0000000103e13fff +MSMapNavigationTelemetryAria arm64  <ccbc8133a7853166838563025d317a3f> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/Frameworks/MSMapNavigationTelemetryAria.framework/MSMapNavigationTelemetryAria
0x0000000103fd4000 -        0x0000000104057fff +AriaObjC arm64  <bdb7a27f41433458b51b38a0378af57b> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/Frameworks/AriaObjC.framework/AriaObjC
0x00000001040d4000 -        0x00000001049e7fff +MicrosoftCognitiveServicesSpeech arm64  <da051163ba4837bbb12f9a2646df918a> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/Frameworks/MicrosoftCognitiveServicesSpeech.framework/MicrosoftCognitiveServicesSpeech
0x0000000104d3c000 -        0x0000000104d57fff +MicrosoftMapsMicrosoftTextToSpeech arm64  <c88b1cafaad639e697545c5be49cddf7> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/Frameworks/MicrosoftMapsMicrosoftTextToSpeech.framework/MicrosoftMapsMicrosoftTextToSpeech
0x0000000104d98000 -        0x0000000105bbffff +MicrosoftMapsNavigation arm64  <976d81dfa64e3143973c9b20e23c1231> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/Frameworks/MicrosoftMapsNavigation.framework/MicrosoftMapsNavigation
0x0000000107dbc000 -        0x0000000107dd7fff +MicrosoftNavigationGPSTrace arm64  <ceb787dcccdc3289a554485d46e9503c> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/Frameworks/MicrosoftNavigationGPSTrace.framework/MicrosoftNavigationGPSTrace
0x0000000107e14000 -        0x0000000107e2bfff +SSZipArchive arm64  <d0d22c511aa935929a0340346a229886> /private/var/containers/Bundle/Application/4C596567-0E15-4BA0-AEA6-CA5C8FA9C08D/XXX.app/Frameworks/SSZipArchive.framework/SSZipArchive
0x0000000192557000 -        0x000000019259bfff  libdispatch.dylib arm64e  <9555d7d99c923af39837ae1f5aa7da44> /usr/lib/system/libdispatch.dylib
0x000000019259c000 -        0x00000001925d8fff  libdyld.dylib arm64e  <ff7d256792773aa6b3dd2c5748e432a6> /usr/lib/system/libdyld.dylib
0x0000000192845000 -        0x0000000192c04fff  CoreFoundation arm64e  <338f58b19b7538a0b908e2574ab1d39a> /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
0x0000000192f74000 -        0x0000000193406fff  CFNetwork arm64e  <df2aa3047ef2311688065072ece34942> /System/Library/Frameworks/CFNetwork.framework/CFNetwork
0x0000000193bc8000 -        0x0000000193e7efff  Foundation arm64e  <4e7d1ff66b6438339e60cc662afe2647> /System/Library/Frameworks/Foundation.framework/Foundation
0x0000000194795000 -        0x0000000195ba0fff  UIKitCore arm64e  <41d9bf6284393b2199594e27e75c294e> /System/Library/PrivateFrameworks/UIKitCore.framework/UIKitCore
0x000000019a384000 -        0x000000019a39bfff  AudioSession arm64e  <1e000caa0b2034debab4ba6000e48199> /System/Library/PrivateFrameworks/AudioSession.framework/AudioSession
0x000000019bcbe000 -        0x000000019bd3dfff  libsystem_c.dylib arm64e  <117637f82c663b93b239a0e23fc95d62> /usr/lib/system/libsystem_c.dylib
0x000000019bd89000 -        0x000000019ce59fff  JavaScriptCore arm64e  <dbc92bff45e83eb0b670612c8cc19b56> /System/Library/Frameworks/JavaScriptCore.framework/JavaScriptCore
0x000000019d92c000 -        0x000000019db9ffff  CoreMotion arm64e  <ba3987d6a1af360aa6c324399bf3d66d> /System/Library/Frameworks/CoreMotion.framework/CoreMotion
0x000000019f534000 -        0x00000001a1adbfff  WebCore arm64e  <dab838aed09e3e3aaadfe0edafcc6902> /System/Library/PrivateFrameworks/WebCore.framework/WebCore
0x00000001a1f36000 -        0x00000001a1fbffff  FrontBoardServices arm64e  <7c0b0af0337138af85345e4bb65286f4> /System/Library/PrivateFrameworks/FrontBoardServices.framework/FrontBoardServices
0x00000001a74ea000 -        0x00000001a7545fff  libc++.1.dylib arm64e  <a0b465d224273dd08577866a3f211ebb> /usr/lib/libc++.1.dylib
0x00000001a9f61000 -        0x00000001a9f69fff  GraphicsServices arm64e  <04a3f8ac0afa3689ab0c613cf170255a> /System/Library/PrivateFrameworks/GraphicsServices.framework/GraphicsServices
0x00000001c0b30000 -        0x00000001c0b62fff  libsystem_kernel.dylib arm64e  <ef6b203fbeed30698623bf884b0f2bad> /usr/lib/system/libsystem_kernel.dylib
0x00000001de5d3000 -        0x00000001de5d9fff  libsystem_platform.dylib arm64e  <987d48d53caa34d79141c0bd6e2351e8> /usr/lib/system/libsystem_platform.dylib
0x00000001de5db000 -        0x00000001de5e6fff  libsystem_pthread.dylib arm64e  <9f849e98f7423b3db01a61f75acf0540> /usr/lib/system/libsystem_pthread.dylib

iCodeWoods avatar Apr 26 '22 03:04 iCodeWoods

@iCodeWoods - this seems to be the scenario where Thread0 is exiting, while Thread2 is still trying to upload the events. Please refer to the use-after-free.md on how to properly shutdown the application without crash.

lalitb avatar Apr 26 '22 14:04 lalitb

We are also observing similar crashes in Teams mobile iOS. Not able to reproduce the crash though. Mainly happening in the logTrace.

Screenshot 2022-05-02 at 11 01 03 AM

cc: @aamittal1989

nishchith-cp avatar May 02 '22 05:05 nishchith-cp

Please refer to comment here : https://github.com/microsoft/cpp_client_telemetry/issues/1012#issuecomment-1115758486

lalitb avatar May 03 '22 05:05 lalitb

@iCodeWoods , @nishchith-cp - Did above suggestion worked, or are you still seeing this issue?

lalitb avatar Jul 07 '22 23:07 lalitb

I was blocked from updating the SDK to the latest due to API issues, Hence couldn't verify this. Will be updating and making these changes this month.

nishchith-cp avatar Jul 08 '22 06:07 nishchith-cp

@tachtevrenidis @iCodeWoods @nishchith-cp - Keeping it open for this week if there is any repro of the issue. As mentioned earlier, if this issue is happening during shutdown, Please refer to the use-after-free.md on how to properly shutdown the application without crash.

lalitb avatar Jul 26 '22 06:07 lalitb

Hey @lalitb

I am still seeing the crashes after making the below changes.

  1. Updated the SDK to the latest version
  2. Calling the below method on app termination (before force quit also) scenarios of the application.
ODWLogManager.flush()
ODWLogManager.applicationWillTerminate() // This inturn calls pauseTransmission and flushAndTeardown

Attaching the reports here. TeamsAppCrash.txt

cc: @aamittal1989

nishchith-cp avatar Aug 08 '22 12:08 nishchith-cp

Hi @lalitb Any updates on this.

nishchith-cp avatar Aug 17 '22 02:08 nishchith-cp

@nishchith-cp - This crash seems to be coming from ongoing requests which can't be stopped/paused using pauseTransmission, and flushAndTearDown. As of now, you can try to wait for random time between pause and teardown. This will reduce the possibility of crash but won't eliminate it. The more complete solution needs to be implemented in v4 release - #822. Please raise a dependency ask to our team for this to be taken care in next semester planning asap. Reach out to our PM Carter Socha for that. I don't see any better solution for this as of now.

lalitb avatar Aug 17 '22 20:08 lalitb

@nishchith-cp - Can you confirm if the latest crash is during app shutdown? In that case, can you ensure that all the ongoing threads are joined to main thread before shutdown, and before invoking flush and applicationWillTerminate from main thread. Looking into the crash dump, there are threads in the application which are still trying to send the event.

lalitb avatar Aug 19 '22 17:08 lalitb

Hey @lalitb

With the proposed fix we still observed crashes during app termination. We added a fix to stop any telemetry being logged on app termination or the app force quit scenarios. Will continue to monitor.

Though we are able to see crashes pointing to OneDS SDK when we force quit the application. The crash log I attached is mainly when the app is being forced to quit by calling exit(0) in scenarios like AppCenter app update, some settings changes, etc.

Below crash on app force quit scenario. report-2517411459120009999-fc42b211-6b47-42a7-9dff-88fea4c5d731.txt

Is there a way we can join the ongoing thread to the main thread? I don't this so. Once the task in the thread is triggered we don't have control over it.

nishchith-cp avatar Aug 23 '22 10:08 nishchith-cp

Is there a way we can join the ongoing thread to the main thread? I don't this so. Once the task in the thread is triggered we don't have control over it.

Why is this not possible. Just curious - Which thread library is used here? In case of force exit, the 1ds sdk library behavior is undefined if there are application threads still trying to send the events.

lalitb avatar Aug 23 '22 17:08 lalitb

@lalitb We are using dispatch_queue and once the task is dispatched we don't have any control over it. We cannot log events on the main thread.

nishchith-cp avatar Sep 02 '22 08:09 nishchith-cp

Hey @lalitb With the updated SDK and the clean-up changes we are seeing around 43.5k crashes on app termination flow with 100% rollout. Can we do something from SDK side to avoid these crashes. Or any more suggestions on app side changes?

We have created below Screenshot 2022-09-13 at 9 06 34 AM

Here is the crash report. report-2517392601050009999-8d6601a2-791f-4378-8cd9-9fcd6a0947b8.txt

Below is the dependency task craeted. https://dev.azure.com/msazure/One/_workitems/edit/15323230 cc: @aamittal1989

nishchith-cp avatar Sep 13 '22 04:09 nishchith-cp

@iCodeWoods Are you still seeing these crashes in AppCenter post the suggested fix?

nishchith-cp avatar Sep 16 '22 06:09 nishchith-cp