DataDog
Crash in DataCompression.swift - Line 85 closure #1 in static Deflate.compress(_:) + 85
Stack trace
Crashed: com.datadoghq.ios-sdk-logging-upload
0 libsystem_kernel.dylib 0xc42c __pthread_kill + 8
1 libsystem_pthread.dylib 0x7c0c pthread_kill + 268
2 libsystem_c.dylib 0x75ba0 abort + 180
3 someapp 0x84f00 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
4 someapp 0x84e64 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
5 someapp 0x84d20 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
6 someapp 0x847cc (Missing UUID 0823cc044ef43811a661c682abd46cdb)
7 someapp 0x84710 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
8 someapp 0x36dc74 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
9 someapp 0x367934 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
10 someapp 0x366d80 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
11 someapp 0x103b70 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
12 someapp 0x10cfb4 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
13 someapp 0x5297e0 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
14 libsystem_pthread.dylib 0x7c0c pthread_kill + 268
15 DriverooIOS-Store 0x14ad58 closure #1 in static Deflate.compress(_:) + 85 (DataCompression.swift:85)
16 DriverooIOS-Store 0x14afa4 specialized __DataStorage.withUnsafeBytes<A>(in:apply:) + 4312362916 (<compiler-generated>:4312362916)
17 DriverooIOS-Store 0x14b4f0 specialized static Deflate.encode(_:) + 30 (DataCompression.swift:30)
18 DriverooIOS-Store 0x14d86c DDURLRequestBuilder.uploadRequest(with:compress:) + 125 (URLRequestBuilder.swift:125)
19 DriverooIOS-Store 0x168800 LoggingRequestBuilder.request(for:with:) + 71 (LoggingV2Configuration.swift:71)
20 DriverooIOS-Store 0x168850 protocol witness for FeatureRequestBuilder.request(for:with:) in conformance LoggingRequestBuilder + 4312483920 (<compiler-generated>:4312483920)
21 DriverooIOS-Store 0x11f40c DataUploader.upload(events:context:) + 30 (DataUploader.swift:30)
22 DriverooIOS-Store 0x11de4c closure #1 in DataUploadWorker.init(queue:fileReader:dataUploader:contextProvider:uploadConditions:delay:featureName:) + 68 (DataUploadWorker.swift:68)
23 DriverooIOS-Store 0x1200c4 thunk for @escaping @callee_guaranteed () -> () + 4312187076 (<compiler-generated>:4312187076)
24 libdispatch.dylib 0x12e5c _dispatch_block_async_invoke2 + 148
25 libdispatch.dylib 0x3dd4 _dispatch_client_callout + 20
26 libdispatch.dylib 0x72d8 _dispatch_continuation_pop + 600
27 libdispatch.dylib 0x1b1c8 _dispatch_source_latch_and_call + 420
28 libdispatch.dylib 0x19d8c _dispatch_source_invoke + 832
29 libdispatch.dylib 0xb284 _dispatch_lane_serial_drain + 368
30 libdispatch.dylib 0xbf64 _dispatch_lane_invoke + 432
31 libdispatch.dylib 0x16cb4 _dispatch_root_queue_drain_deferred_wlh + 288
32 libdispatch.dylib 0x16528 _dispatch_workloop_worker_thread + 404
33 libsystem_pthread.dylib 0x4934 _pthread_wqthread + 288
34 libsystem_pthread.dylib 0x10cc start_wqthread + 8
Reproduction steps
We haven't been able to reproduce this yet.
Volume
2%
Affected SDK versions
Latest
Latest working SDK version
Latest
Does the crash manifest in the latest SDK version?
Yes
Deployment Target
iOS 15
Device Information
Seems to happen across all devices and OS's distributed with usage.
Other relevant information
No response
Hey @iainsmith 👋
Thank you for the report.
Just to confirm, in which version this crash first appeared? is it on the 2.15.0 (latest) only?
Do you have by any chance the exception code and description?
Thanks!
Hi @maxep! We have this crash on a version 1.24.0 (we plan to update to 2.14.1 this week), but I can't see any changes in files that are mentioned in the stack trace between the versions.
The crash happens a few seconds after our application starts in Deflate.compress.
Actually, this is a group of crashes, the majority of them marked as SIGABRT (ABORT), others EXC_BAD_ACCESS KERN_INVALID_ADDRESS.
Alternative stack trace
Crashed: com.datadoghq.ios-sdk-tracing-upload
0 libsystem_kernel.dylib 0xc254 __pthread_kill + 8
1 libsystem_pthread.dylib 0x7ef8 pthread_kill + 268
2 libsystem_c.dylib 0x77ad8 abort + 128
3 someapp 0x84f00 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
4 someapp 0x84e64 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
5 someapp 0x84d20 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
6 someapp 0x847cc (Missing UUID 0823cc044ef43811a661c682abd46cdb)
7 someapp 0x84710 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
8 someapp 0x36dc74 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
9 someapp 0x367934 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
10 someapp 0x366d80 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
11 someapp 0x103b70 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
12 someapp 0x10cfb4 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
13 someapp 0x5297e0 (Missing UUID 0823cc044ef43811a661c682abd46cdb)
14 libsystem_pthread.dylib 0x7ef8 pthread_kill + 268
15 DriverooIOS-Store 0x147100 closure #1 in static Deflate.compress(_:) + 85 (DataCompression.swift:85)
16 DriverooIOS-Store 0x14734c specialized __DataStorage.withUnsafeBytes<A>(in:apply:) + 4371149644 (<compiler-generated>:4371149644)
17 DriverooIOS-Store 0x147898 specialized static Deflate.encode(_:) + 30 (DataCompression.swift:30)
18 DriverooIOS-Store 0x149c14 DDURLRequestBuilder.uploadRequest(with:compress:) + 125 (URLRequestBuilder.swift:125)
19 DriverooIOS-Store 0x2525f4 TracingRequestBuilder.request(for:with:) + 64 (TracingV2Configuration.swift:64)
20 DriverooIOS-Store 0x252644 protocol witness for FeatureRequestBuilder.request(for:with:) in conformance TracingRequestBuilder + 4372244036 (<compiler-generated>:4372244036)
21 DriverooIOS-Store 0x11b7b4 DataUploader.upload(events:context:) + 30 (DataUploader.swift:30)
22 DriverooIOS-Store 0x11a1f4 closure #1 in DataUploadWorker.init(queue:fileReader:dataUploader:contextProvider:uploadConditions:delay:featureName:) + 68 (DataUploadWorker.swift:68)
23 DriverooIOS-Store 0x11c46c thunk for @escaping @callee_guaranteed () -> () + 4370973804 (<compiler-generated>:4370973804)
24 libdispatch.dylib 0x132f0 _dispatch_block_async_invoke2 + 148
25 libdispatch.dylib 0x40d8 _dispatch_client_callout + 20
26 libdispatch.dylib 0x7588 _dispatch_continuation_pop + 596
27 libdispatch.dylib 0x1b53c _dispatch_source_latch_and_call + 420
28 libdispatch.dylib 0x1a104 _dispatch_source_invoke + 836
29 libdispatch.dylib 0xb568 _dispatch_lane_serial_drain + 368
30 libdispatch.dylib 0xc21c _dispatch_lane_invoke + 432
31 libdispatch.dylib 0x17258 _dispatch_root_queue_drain_deferred_wlh + 288
32 libdispatch.dylib 0x16aa4 _dispatch_workloop_worker_thread + 532
33 libsystem_pthread.dylib 0x4c7c _pthread_wqthread + 288
34 libsystem_pthread.dylib 0x1488 start_wqthread + 8
Hi @dasha-frolova-roo,
Thank you for the additional info. At this point, we call the compression_encode_buffer from the Compression framework.
Are you missing the symbols (dSYM) for the binary with UUID 0823cc044ef43811a661c682abd46cdb, it would be interesting to see the call stack from someapp. It seems that someapp is calling the abort (at frame 3) which could be caused by a runtime exception. Do you have by any chance the Last Exception Backtrace section of the crash?
Also, is this crash happening only on iOS 15 as mentioned in this issue? if not, what's the range of OSs affected?
Thanks!
This is happening on all versions of iOS. Unfortunately I can't provide the rest of the symbols.
We can't retrieve the rest of the symbols, unfortunately, but I have some more info from .crash files. Here are some variants
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Codes: 0x0000000000000001, 0x0000000000000000
VM Region Info: 0 is not in any region. Bytes before following region: 4334338048
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
--->
__TEXT 10258c000-102594000 [ 32K] r-x/r-x SM=COW /var/containers/Bundle/Application/A4ACCFFE-943C-4A64-8A15-94EE2770172B/someapp.app/someapp
Termination Reason: SIGNAL 11 Segmentation fault: 11
Terminating Process: exc handler [3551]
and
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x000063a7bb1a1380 -> 0x00000027bb1a1380 (possible pointer authentication failure)
Exception Codes: 0x0000000000000001, 0x000063a7bb1a1380
VM Region Info: 0x27bb1a1380 is in 0x1000000000-0x7000000000; bytes after start: 101923296128 bytes before end: 310393564287
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
commpage (reserved) fc0000000-1000000000 [ 1.0G] ---/--- SM=NUL ...(unallocated)
---> GPU Carveout (reserved) 1000000000-7000000000 [384.0G] ---/--- SM=NUL ...(unallocated)
UNUSED SPACE AT END
Termination Reason: SIGNAL 11 Segmentation fault: 11
Terminating Process: exc handler [9160]
Thread 8 crashed with ARM Thread State (64-bit):
x0: 0x0000000000000000 x1: 0x00000003011cda80 x2: 0x0000000000000070 x3: 0x0000000000000070
x4: 0x0000000000000003 x5: 0x0000000000000c00 x6: 0x00000003011ccc00 x7: 0x000000004694a9f2
x8: 0x000000000000002f x9: 0x0000000000000000 x10: 0x0000000000000000 x11: 0x0000000000000080
x12: 0x0000000303d33344 x13: 0x00000000001ff800 x14: 0x00000000000007fb x15: 0x00000000e9a0f819
x16: 0x0000000000000018 x17: 0x00000000e9c0f016 x18: 0x0000000000000000 x19: 0x0000000303b773b8
x20: 0x0000000303b773a0 x21: 0x00000003030acf18 x22: 0x0aaaaaaaaaaaaaaa x23: 0x0555555555555555
x24: 0x000000002b6f9e21 x25: 0x000000000d4c0eac x26: 0x000000005f3b768f x27: 0x0000000105c7d000
x28: 0x0000000105c7d000 fp: 0x000000016e0a1340 lr: 0x0000000105458ce4
sp: 0x000000016e0a12f0 pc: 0x0000000105458df0 cpsr: 0x80000000
esr: 0x92000046 (Data Abort) byte write Translation fault
Another scenario
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: SIGNAL 6 Abort trap: 6
Terminating Process: someapp [32539]
Thread 7 crashed with ARM Thread State (64-bit):
x0: 0x0000000000000000 x1: 0x0000000000000000 x2: 0x0000000000000000 x3: 0x0000000000000000
x4: 0x0000000000000000 x5: 0x0000000000989680 x6: 0x0000000300cb6400 x7: 0x000000004694a9f2
x8: 0x790c863a02f30752 x9: 0x790c863b6f05b752 x10: 0x00000000000003e8 x11: 0x0000000000000080
x12: 0x000000030205ed30 x13: 0x00000000001ff800 x14: 0x00000000000007fb x15: 0x00000000c822a849
x16: 0x0000000000000148 x17: 0x000000016df6b000 x18: 0x0000000000000000 x19: 0x0000000000000006
x20: 0x0000000000007403 x21: 0x000000016df6b0e0 x22: 0x0aaaaaaaaaaaaaaa x23: 0x0555555555555555
x24: 0x000000002b6f9e21 x25: 0x000000000d4c0eac x26: 0x000000005f3b768f x27: 0x0000000105bb5000
x28: 0x0000000105bb5000 fp: 0x000000016df691e0 lr: 0x00000001fe61ec0c
sp: 0x000000016df691c0 pc: 0x00000001ea87f42c cpsr: 0x40001000
esr: 0x56000080 Address size fault
Thanks @dasha-frolova-roo, that's very helpful.
It seems that the abort is called at frame 4 by BrazeUIMocks.swift:53. Looks like a crash in a test environment, no?
My assumption is that the Datadog compression operation is suspended to perform BrazeUIMocks drawing, which abort the application somehow.
Hi @maxep,
Unfortunately, this symbolification wasn't done with the right dSYM, and we don't have access to the correct one. So please ignore it.
We updated Datadog to the version 2.14.1 but the crashes are still there. The crashes started very suddenly in one specific version of our app so at the moment we're reverting changes in that version one by one to see if it helps.
Hey @dasha-frolova-roo 👋
The crashes started very suddenly in one specific version of our app
Did you upgrade the Datadog SDK in that specific version of your app? If so, I would be interested to know the previous and new versions of the SDK.
Hey @maxep,
That could explain a lot, but no, we didn't update Datadog in that version of our app. The Datadog SDK version at the time was 1.24.0. We updated it to 2.14.1 after the crashes started in hope to fix them, but it didn't help.
Hey @dasha-frolova-roo , @iainsmith 👋
I'm closing this issue as I don't think this is due to the SDK, the crash was not introduced by a SDK update.
The compression is a core mechanism of the Datadog SDK so we would have had more report if it was crashing. I suspect it has something to do with the someapp binary we see in your stacktraces.
Feel free to comment to this issue if you make more findings, you can also open a ticket to our support if you need to share private info.
The issue was indeed in someapp, it's another third-party framework that we're using, and it had some thread safety problems that caused a race condition. They are fixing it now, so it's a happy end.
Sorry we bothered you and thanks for the help!
Thanks for the heads up, @dasha-frolova-roo 👍 Nice to hear that you found the issue. And thank you for using our SDK!