YapDatabase icon indicating copy to clipboard operation
YapDatabase copied to clipboard

reabstraction thunk helper swift_getObjectType crash

Open chrisballinger opened this issue 6 years ago • 8 comments

I think there was a regression introduced by Xcode 9.4's Swift compiler. The below code was unchanged between releases, with the only difference being the Swift compiler version. This is not necessarily a bug in YapDatabase itself, but in case anyone else is seeing this in their crash logs, the "fix" seems to be disable the code completely until Swift 4.2 is released.

        guard let secondaryIndexTransaction = self.ext(SecondaryIndexName.mediaItems) as? YapDatabaseSecondaryIndexTransaction else {
            return []
        }
        var unfinished: [OTRMediaItem] = []
        let queryString = "Where \(MediaItemIndexColumnName.transferProgress) < 1 AND \(MediaItemIndexColumnName.isIncoming) == 1"
        let query = YapDatabaseQuery(string: queryString, parameters: [])
        secondaryIndexTransaction.enumerateKeysAndObjects(matching: query) { (key, collection, object, stop) in
            if let download = object as? OTRMediaItem {
                unfinished.append(download)
            } else {
                DDLogError("Non-media item object in downloads index \(object)")
            }
        }
        return unfinished

It crashes with an error in the Swift runtime swift_getObjectType, which I believe is related to this Swift bug: https://bugs.swift.org/browse/SR-7719

The best part is that I'm calling this code on app launch, so all of my users have a crash on launch bug now. Good way to spend a Saturday morning!

Thread 6 Crashed:
0   libswiftCore.dylib              0x0000000104343a94 0x104088000 + 2865812 (swift_getObjectType + 60)
1   ChatSecureCore                  0x0000000103b74824 reabstraction thunk helper from @escaping @callee_guaranteed (@owned Swift.String, @owned Swift.String, @in Any, @unowned Swift.UnsafeMutablePointer<ObjectiveC.ObjCBool>) -> () to @escaping @callee_unowned @convention(block) (@unowned __C.NSString, @unowned __C.NSString, @unowned Swift.AnyObject, @unowned Swift.UnsafeMutablePointer<ObjectiveC.ObjCBool>) -> () (YapDatabaseTransaction+ChatSecure.swift:0)
2   ChatSecureCore                  0x0000000103b74824 reabstraction thunk helper from @escaping @callee_guaranteed (@owned Swift.String, @owned Swift.String, @in Any, @unowned Swift.UnsafeMutablePointer<ObjectiveC.ObjCBool>) -> () to @escaping @callee_unowned @convention(block) (@unowned __C.NSString, @unowned __C.NSString, @unowned Swift.AnyObject, @unowned Swift.UnsafeMutablePointer<ObjectiveC.ObjCBool>) -> () (YapDatabaseTransaction+ChatSecure.swift:0)
3   YapDatabase                     0x00000001035e9a34 __88-[YapDatabaseSecondaryIndexTransaction enumerateKeysAndObjectsMatchingQuery:usingBlock:]_block_invoke (YapDatabaseSecondaryIndexTransaction.m:1290)
4   YapDatabase                     0x00000001035e92e4 -[YapDatabaseSecondaryIndexTransaction _enumerateRowidsMatchingQuery:usingBlock:] (YapDatabaseSecondaryIndexTransaction.m:1211)
5   YapDatabase                     0x00000001035e9938 -[YapDatabaseSecondaryIndexTransaction enumerateKeysAndObjectsMatchingQuery:usingBlock:] (YapDatabaseSecondaryIndexTransaction.m:1278)
6   ChatSecureCore                  0x0000000103b74bdc (extension in ChatSecureCore):__C.YapDatabaseReadTransaction.unfinishedDownloads() -> [__C.OTRMediaItem] (YapDatabaseTransaction+ChatSecure.swift:158)
7   ChatSecureCore                  0x0000000103aefeb8 function signature specialization <Arg[0] = Owned To Guaranteed> of closure #1 (__C.YapDatabaseReadTransaction) -> () in ChatSecureCore.FileTransferManager.resumeDownloads() -> () (FileTransferManager.swift:159)
8   ChatSecureCore                  0x0000000103aef8ac partial apply forwarder for closure #1 (__C.YapDatabaseReadTransaction) -> () in ChatSecureCore.FileTransferManager.resumeDownloads() -> () (FileTransferManager.swift:0)
9   ChatSecureCore                  0x0000000103b74794 reabstraction thunk helper from @escaping @callee_guaranteed (@owned __C.YapDatabaseReadTransaction) -> () to @escaping @callee_unowned @convention(block) (@unowned __C.YapDatabaseReadTransaction) -> () (YapDatabaseTransaction+ChatSecure.swift:0)
10  YapDatabase                     0x0000000103590f5c __76-[YapDatabaseConnection asyncReadWithBlock:completionQueue:completionBlock:]_block_invoke (YapDatabaseConnection.m:2222)
11  libdispatch.dylib               0x0000000184734aa0 _dispatch_call_block_and_release + 20
12  libdispatch.dylib               0x0000000184734a60 _dispatch_client_callout + 12
13  libdispatch.dylib               0x000000018473e9b4 _dispatch_queue_serial_drain$VARIANT$mp + 604
14  libdispatch.dylib               0x000000018473f2fc _dispatch_queue_invoke$VARIANT$mp + 332
15  libdispatch.dylib               0x000000018473fcc8 _dispatch_root_queue_drain_deferred_wlh$VARIANT$mp + 336
16  libdispatch.dylib               0x0000000184748098 _dispatch_workloop_worker_thread$VARIANT$mp + 664
17  libsystem_pthread.dylib         0x0000000184a67e70 _pthread_wqthread + 856

https://github.com/ChatSecure/ChatSecure-iOS/issues/1034

chrisballinger avatar Jul 14 '18 15:07 chrisballinger

Another trace in a different place YapDatabaseAutoViewTransaction:

Thread 9 Crashed:
0   libswiftCore.dylib              0x0000000102b27a94 0x10286c000 + 2865812 (swift_getObjectType + 60)
1   YapTaskQueue                    0x0000000101fb2c00 reabstraction thunk helper from @escaping @callee_guaranteed (@owned __C.YapDatabaseReadTransaction, @owned Swift.String, @owned Swift.String, @owned Swift.String, @in Any, @owned Swift.String, @owned Swift.String, @in Any) -> (@unowned __C_Synthesized.ComparisonResult) to @escaping @callee_unowned @convention(block) (@unowned __C.YapDatabaseReadTransaction, @unowned __C.NSString, @unowned __C.NSString, @unowned __C.NSString, @unowned Swift.AnyObject, @unowned __C.NSString, @unowned __C.NSString, @unowned Swift.AnyObject) -> (@unowned __C_Synthesized.ComparisonResult) (YapTaskQueueMasterBroker.swift:0)
2   YapTaskQueue                    0x0000000101fb2c00 reabstraction thunk helper from @escaping @callee_guaranteed (@owned __C.YapDatabaseReadTransaction, @owned Swift.String, @owned Swift.String, @owned Swift.String, @in Any, @owned Swift.String, @owned Swift.String, @in Any) -> (@unowned __C_Synthesized.ComparisonResult) to @escaping @callee_unowned @convention(block) (@unowned __C.YapDatabaseReadTransaction, @unowned __C.NSString, @unowned __C.NSString, @unowned __C.NSString, @unowned Swift.AnyObject, @unowned __C.NSString, @unowned __C.NSString, @unowned Swift.AnyObject) -> (@unowned __C_Synthesized.ComparisonResult) (YapTaskQueueMasterBroker.swift:0)
3   YapDatabase                     0x0000000101d51614 __102-[YapDatabaseAutoViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:]_block_invoke (YapDatabaseAutoViewTransaction.m:0)
4   YapDatabase                     0x0000000101d51154 -[YapDatabaseAutoViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:] (YapDatabaseAutoViewTransaction.m:706)
5   YapDatabase                     0x0000000101d51da0 -[YapDatabaseAutoViewTransaction _handleChangeWithRowid:collectionKey:object:metadata:grouping:sorting:blockInvokeBitMask:changesBitMask:isInsert:] (YapDatabaseAutoViewTransaction.m:878)
6   YapDatabase                     0x0000000101d51ec4 -[YapDatabaseAutoViewTransaction didInsertObject:forCollectionKey:withMetadata:rowid:] (YapDatabaseAutoViewTransaction.m:910)
7   YapDatabase                     0x0000000101df49fc -[YapDatabaseReadWriteTransaction setObject:forKey:inCollection:withMetadata:serializedObject:serializedMetadata:] (YapDatabaseTransaction.m:0)
8   YapDatabase                     0x0000000101df3cd8 -[YapDatabaseReadWriteTransaction setObject:forKey:inCollection:withMetadata:] (YapDatabaseTransaction.m:4680)
9   ChatSecureCore                  0x00000001021ceaf8 -[OTRYapDatabaseObject saveWithTransaction:] (OTRYapDatabaseObject.m:32)
10  ChatSecureCore                  0x00000001021efca0 __34-[OTRXMPPManager enqueueMessages:]_block_invoke_2 (OTRXMPPManager.m:593)
11  CoreFoundation                  0x0000000182de3f38 -[__NSSingleObjectArrayI enumerateObjectsWithOptions:usingBlock:] + 72
12  ChatSecureCore                  0x00000001021efbd0 __34-[OTRXMPPManager enqueueMessages:]_block_invoke (OTRXMPPManager.m:583)
13  YapDatabase                     0x0000000101d89408 __81-[YapDatabaseConnection asyncReadWriteWithBlock:completionQueue:completionBlock:]_block_invoke_2 (YapDatabaseConnection.m:2335)
14  libdispatch.dylib               0x000000018273cae4 _dispatch_client_callout + 12
15  libdispatch.dylib               0x0000000182745640 _dispatch_queue_barrier_sync_invoke_and_complete + 52
16  YapDatabase                     0x0000000101d8933c __81-[YapDatabaseConnection asyncReadWriteWithBlock:completionQueue:completionBlock:]_block_invoke (YapDatabaseConnection.m:2330)
17  libdispatch.dylib               0x000000018273cb24 _dispatch_call_block_and_release + 20
18  libdispatch.dylib               0x000000018273cae4 _dispatch_client_callout + 12
19  libdispatch.dylib               0x0000000182746a38 _dispatch_queue_serial_drain$VARIANT$mp + 604
20  libdispatch.dylib               0x0000000182747380 _dispatch_queue_invoke$VARIANT$mp + 332
21  libdispatch.dylib               0x0000000182747d4c _dispatch_root_queue_drain_deferred_wlh$VARIANT$mp + 336
22  libdispatch.dylib               0x000000018275011c _dispatch_workloop_worker_thread$VARIANT$mp + 664
23  libsystem_pthread.dylib         0x0000000182a6fe70 _pthread_wqthread + 856

chrisballinger avatar Jul 15 '18 20:07 chrisballinger

So from what I understand about SR-7719, this can happen when the bridged Any is unexpectedly nil. It looks like it's possible, although unlikely, for some of these Anys to be nil. For example right before the block callback is called here:

- (BOOL)enumerateRowsMatchingQuery:(YapDatabaseQuery *)query
                        usingBlock:
                            (void (^)(NSString *collection, NSString *key, id object, id metadata, BOOL *stop))block
{
	BOOL result = [self _enumerateRowidsMatchingQuery:query usingBlock:^(int64_t rowid, BOOL *stop) {
		
		if (block == NULL) // Query test : caller still wants BOOL result
		{
			*stop = YES;
			return; // from block
		}
		
		YapCollectionKey *ck = nil;
		id object = nil;
		id metadata = nil;
		[self->databaseTransaction getCollectionKey:&ck object:&object metadata:&metadata forRowid:rowid];


		// *** if object is nil, this will crash the Swift runtime shipped with Xcode 9.3 and 9.4
		block(ck.collection, ck.key, object, metadata, stop);
	}];
	
	return result;
}

Perhaps it would be good to add nil checks there, even if the crash is fixed in Xcode 10.

chrisballinger avatar Jul 19 '18 02:07 chrisballinger

Hi @chrisballinger

Thank you for debugging this! I've been experiencing what looks like the same bug in my app. Hopefully, it only seems to happen sporadically and only few users have been affected so far (although these users generate an awful number of crashes).

Here are the traces in my case: https://gist.github.com/flovouin/8f5dd3cce5f842bde514c59256fc20fc https://gist.github.com/flovouin/e1b0985facb6963bfe16de46ac777c2c

Could you please confirm that your PR #463 fixes this? I'm already maintaining a fork of YapDatabase for other reasons, so I don't mind cherry-picking your commits if they do solve the problem, in the case your PR wouldn't be merged quickly. :-)

Also, by the looks of the second trace I shared, it would seem that another check is necessary in YapDatabaseAutoViewTransaction's findRangeInGroup. If I'm not mistaken, your PR doesn't add a check there. ;-)

Thanks again for your work and I hope your Saturday mornings will be more quiet in the future.

Flo

flovouin avatar Jul 26 '18 15:07 flovouin

screen shot 2018-07-27 at 8 13 27 am

@flovouin Fewer total (all cause) crashes in the app after the patch, but looks like there are still more places that need nil checks.

You're right, needs one in findRangeInGroup too. Looks like I might need one in readWithBlock and readWriteWithBlock?

Thread 0 Crashed:
0   libswiftUIKit.dylib                  0x000000010348e1b0 0x103484000 + 41392
1   ChatSecureCore                       0x0000000102829e00 function signature specialization <Arg[0] = Owned To Guaranteed> of closure #1 (__ObjC.YapDatabaseReadTransaction) -> () in ChatSecureCore.OTRYapViewHandler.object(Foundation.IndexPath) -> Swift.AnyObject? (OTRYapViewHandler.swift:133)
2   ChatSecureCore                       0x0000000102829f74 partial apply forwarder for closure #1 (__ObjC.YapDatabaseReadTransaction) -> () in ChatSecureCore.OTRYapViewHandler.object(Foundation.IndexPath) -> Swift.AnyObject? (OTRYapViewHandler.swift:0)
3   ChatSecureCore                       0x000000010289dfcc reabstraction thunk helper from @escaping @callee_guaranteed (@owned __ObjC.YapDatabaseReadTransaction) -> () to @escaping @callee_unowned @convention(block) (@unowned __ObjC.YapDatabaseReadTransaction) -> () (YapDatabaseTransaction+ChatSecure.swift:0)
4   YapDatabase                          0x00000001022b81d4 __39-[YapDatabaseConnection readWithBlock:]_block_invoke (YapDatabaseConnection.m:2028)
5   libdispatch.dylib                    0x0000000182114a60 _dispatch_client_callout + 12
6   libdispatch.dylib                    0x000000018211d5bc _dispatch_queue_barrier_sync_invoke_and_complete + 52
7   YapDatabase                          0x00000001022b8178 -[YapDatabaseConnection readWithBlock:] (YapDatabaseConnection.m:2015)
8   ChatSecureCore                       0x0000000102827b64 function signature specialization <Arg[0] = Owned To Guaranteed> of ChatSecureCore.OTRYapViewHandler.object(Foundation.IndexPath) -> Swift.AnyObject? (OTRYapViewHandler.swift:124)
9   ChatSecureCore                       0x0000000102825a7c @objc ChatSecureCore.OTRYapViewHandler.object(Foundation.IndexPath) -> Swift.AnyObject? (OTRYapViewHandler.swift:0)
10  ChatSecureCore                       0x0000000102741824 -[OTRMessagesViewController messageAtIndexPath:] (OTRMessagesViewController.m:1193)
11  ChatSecureCore                       0x0000000102746f30 -[OTRMessagesViewController deleteMessageAtIndexPath:] (OTRMessagesViewController.m:2012)
12  ChatSecureCore                       0x0000000102743a1c -[OTRMessagesViewController collectionView:performAction:forItemAtIndexPath:withSender:] (OTRMessagesViewController.m:1521)
13  ChatSecureCore                       0x00000001026cd844 -[JSQMessagesCollectionViewCell(ChatSecure) performSelectorOnParentCollectionView:withSender:] (JSQMessagesCollectionViewCell+ChatSecure.m:46)
14  UIKit                                0x000000018cbbc254 -[UICalloutBar buttonPressed:] + 412
15  UIKit                                0x000000018cbbb4e0 -[UICalloutBarButton fadeAndSendAction] + 80
16  Foundation                           0x00000001832584ac __NSFireDelayedPerform + 404
17  CoreFoundation                       0x00000001827cbaa8 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 24
18  CoreFoundation                       0x00000001827cb76c __CFRunLoopDoTimer + 860
19  CoreFoundation                       0x00000001827cb010 __CFRunLoopDoTimers + 244
20  CoreFoundation                       0x00000001827c8b60 __CFRunLoopRun + 2164
21  CoreFoundation                       0x00000001826e8da8 CFRunLoopRunSpecific + 548
22  GraphicsServices                     0x00000001846ce020 GSEventRunModal + 96
23  UIKit                                0x000000018c708758 UIApplicationMain + 232
24  ChatSecure                           0x0000000100e3fd20 main (AppDelegate.swift:14)
25  libdyld.dylib                        0x0000000182179fc0 start + 0

readWriteWithBlock?

Thread 9 Crashed:
0   libobjc.A.dylib                      0x000000018278c910 objc_msgSend + 16
1   Foundation                           0x0000000184007ddc _NSDescriptionWithStringProxyFunc + 60
2   CoreFoundation                       0x000000018357cc70 __CFStringAppendFormatCore + 8828
3   CoreFoundation                       0x000000018357e740 _CFStringCreateWithFormatAndArgumentsAux2 + 128
4   CocoaLumberjack                      0x0000000104d1ea38 +[DDLog log:level:flag:context:file:function:line:tag:format:] (DDLog.m:385)
5   YapDatabase                          0x0000000105844694 -[YapDatabaseViewTransaction insertRowid:collectionKey:inGroup:atIndex:] (YapDatabaseViewTransaction.m:1315)
6   YapDatabase                          0x0000000105784b60 -[YapDatabaseAutoViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:] (YapDatabaseAutoViewTransaction.m:727)
7   YapDatabase                          0x0000000105785748 -[YapDatabaseAutoViewTransaction _handleChangeWithRowid:collectionKey:object:metadata:grouping:sorting:blockInvokeBitMask:changesBitMask:isInsert:] (YapDatabaseAutoViewTransaction.m:886)
8   YapDatabase                          0x000000010578586c -[YapDatabaseAutoViewTransaction didInsertObject:forCollectionKey:withMetadata:rowid:] (YapDatabaseAutoViewTransaction.m:918)
9   YapDatabase                          0x0000000105828578 -[YapDatabaseReadWriteTransaction setObject:forKey:inCollection:withMetadata:serializedObject:serializedMetadata:] (YapDatabaseTransaction.m:0)
10  YapDatabase                          0x0000000105827828 -[YapDatabaseReadWriteTransaction setObject:forKey:inCollection:withMetadata:] (YapDatabaseTransaction.m:4694)
11  ChatSecureCore                       0x0000000105c043b0 -[OTRYapDatabaseObject saveWithTransaction:] (OTRYapDatabaseObject.m:32)
12  ChatSecureCore                       0x0000000105d27f9c partial apply forwarder for closure #1 (__C.YapDatabaseReadWriteTransaction) -> () in ChatSecureCore.FileTransferManager.(continueDownloading in _462584AE3147906C3CA5C857C5C810B1)(downloadMessage: __C.OTRDownloadMessage, url: Foundation.URL, contentType: Swift.String?, contentLength: Swift.UInt) -> () (FileTransferManager.swift:0)
13  ChatSecureCore                       0x0000000105cb6fb4 reabstraction thunk helper from @escaping @callee_guaranteed (@owned __C.YapDatabaseReadWriteTransaction) -> () to @escaping @callee_unowned @convention(block) (@unowned __C.YapDatabaseReadWriteTransaction) -> () (OTRStreamManagementDelegate.swift:0)
14  YapDatabase                          0x00000001057bc56c __44-[YapDatabaseConnection readWriteWithBlock:]_block_invoke_2 (YapDatabaseConnection.m:2120)
15  libdispatch.dylib                    0x0000000182eb0ae4 _dispatch_client_callout + 12
16  libdispatch.dylib                    0x0000000182eb9640 _dispatch_queue_barrier_sync_invoke_and_complete + 52
17  YapDatabase                          0x00000001057bc4b0 __44-[YapDatabaseConnection readWriteWithBlock:]_block_invoke (YapDatabaseConnection.m:2115)
18  libdispatch.dylib                    0x0000000182eb0ae4 _dispatch_client_callout + 12
19  libdispatch.dylib                    0x0000000182ebf110 _dispatch_sync_invoke_and_complete_recurse + 60
20  libdispatch.dylib                    0x0000000182ebec00 _dispatch_sync_wait + 516
21  YapDatabase                          0x00000001057bc344 -[YapDatabaseConnection readWriteWithBlock:] (YapDatabaseConnection.m:2089)
22  ChatSecureCore                       0x0000000105d24238 function signature specialization <Arg[0] = Owned To Guaranteed, Arg[1] = Owned To Guaranteed, Arg[2] = Owned To Guaranteed, Arg[3] = Dead> of ChatSecureCore.FileTransferManager.(continueDownloading in _462584AE3147906C3CA5C857C5C810B1)(downloadMessage: __C.OTRDownloadMessage, url: Foundation.URL, contentType: Swift.String?, contentLength: Swift.UInt) -> () (FileTransferManager.swift:662)
23  ChatSecureCore                       0x0000000105d24ffc function signature specialization <Arg[0] = Owned To Guaranteed> of closure #1 (Alamofire.DataResponse<Foundation.Data>) -> () in closure #1 ([__C.URLSessionDataTask], [__C.URLSessionUploadTask], [__C.URLSessionDownloadTask]) -> () in ChatSecureCore.FileTransferManager.(downloadMedia in _462584AE3147906C3CA5C857C5C810B1)(__C.OTRDownloadMessage) -> () (FileTransferManager.swift:0)
24  ChatSecureCore                       0x0000000105d28004 merged partial apply forwarder for closure #1 (Alamofire.DataResponse<Foundation.Data>) -> () in closure #1 ([__C.URLSessionDataTask], [__C.URLSessionUploadTask], [__C.URLSessionDownloadTask]) -> () in ChatSecureCore.FileTransferManager.(downloadMedia in _462584AE3147906C3CA5C857C5C810B1)(__C.OTRDownloadMessage) -> () + 28
25  Alamofire                            0x00000001043d4afc partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@owned Alamofire.DataResponse<Foundation.Data>) -> () to @escaping @callee_guaranteed (@in Alamofire.DataResponse<Foundation.Data>) -> () (ResponseSerialization.swift:0)
26  Alamofire                            0x00000001043cfdac generic specialization <Alamofire.DataResponseSerializer<Foundation.Data>> of closure #1 () -> () in closure #1 () -> () in Alamofire.DataRequest.response<A where A: Alamofire.DataResponseSerializerProtocol>(queue: __C.DispatchQueue?, responseSerializer: A, completionHandler: (Alamofire.DataResponse<A.SerializedObject>) -> ()) -> Self (ResponseSerialization.swift:167)
27  Alamofire                            0x00000001043b24c0 reabstraction thunk helper from @escaping @callee_guaranteed () -> () to @escaping @callee_unowned @convention(block) () -> () (NetworkReachabilityManager.swift:0)
28  libdispatch.dylib                    0x0000000182eb0b24 _dispatch_call_block_and_release + 20
29  libdispatch.dylib                    0x0000000182eb0ae4 _dispatch_client_callout + 12
30  libdispatch.dylib                    0x0000000182ebaa38 _dispatch_queue_serial_drain$VARIANT$mp + 604
31  libdispatch.dylib                    0x0000000182ebb380 _dispatch_queue_invoke$VARIANT$mp + 332
32  libdispatch.dylib                    0x0000000182ebbd4c _dispatch_root_queue_drain_deferred_wlh$VARIANT$mp + 336
33  libdispatch.dylib                    0x0000000182ec411c _dispatch_workloop_worker_thread$VARIANT$mp + 664
34  libsystem_pthread.dylib              0x00000001831e3e70 _pthread_wqthread + 856
35  libsystem_pthread.dylib              0x00000001831e3b08 start_wqthread + 0

Feel free to submit a PR to my PR! :)

chrisballinger avatar Jul 27 '18 15:07 chrisballinger

Thanks for your reply!

Strangely these bugs have only affected a very limited number of users in my app, but are still concerning. I'm on leave for a bit more than a week and unfortunately I won't have the chance to take a more thorough look before I come back. I'll keep you posted when I dig more into this issue.

flovouin avatar Jul 31 '18 11:07 flovouin

The problem is still actual.

AKoulabukhov avatar Mar 22 '21 10:03 AKoulabukhov

Hello. In our company we have the same issue, I have studied it a bit along with already mentioned swift bug. The crash happens only in case the device has insufficient disk space.

Trace

Thread 12 Crashed:
0   libswiftCore.dylib                   0x00000001e940c92c  swift_getObjectType + 2939180
1   Fiesta                               0x000000010134428c  thunk for @escaping @callee_guaranteed (@guaranteed YapDatabaseReadTransaction, @guaranteed String, @guaranteed String, @guaranteed String, @in_guaranteed Any, @guaranteed String, @guaranteed String, @in_guaranteed Any) -> (@unowned NSComparisonResult) (<compiler-generated>:0)
2   YapDatabase                          0x0000000105760ba8  __102-[YapDatabaseAutoViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:]_block_invoke (YapDatabaseAutoViewTransaction.m:0)
3   YapDatabase                          0x00000001057606fc  -[YapDatabaseAutoViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:] (YapDatabaseAutoViewTransaction.m:708)
4   YapDatabase                          0x0000000105761300  -[YapDatabaseAutoViewTransaction _handleChangeWithRowid:collectionKey:object:metadata:grouping:sorting:blockInvokeBitMask:changesBitMask:isInsert:] (YapDatabaseAutoViewTransaction.m:885)
5   YapDatabase                          0x0000000105761424  -[YapDatabaseAutoViewTransaction didInsertObject:forCollectionKey:withMetadata:rowid:] (YapDatabaseAutoViewTransaction.m:0)
6   YapDatabase                          0x00000001057d370c  -[YapDatabaseReadWriteTransaction setObject:forKey:inCollection:withMetadata:serializedObject:serializedMetadata:] (YapDatabaseTransaction.m:4945)
7   YapDatabase                          0x00000001057d29dc  -[YapDatabaseReadWriteTransaction setObject:forKey:inCollection:] (YapDatabaseTransaction.m:0)

How the crash happens According to comments in swift bug, the crash happens when swift tries to get type of an object in case it is not an object but nil. It is described in better words in this comment So if swift's block wrapper (thunk for @escaping @callee_guaranteed ...) gets nil for parameter with type Any it crashes.

I created a small app to demonstrate the crash. image

In our case the crash happens in this part of [YapDatabaseAutoViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:] (lines 600 to 614)

else if (sorting->blockType == YapDatabaseBlockTypeWithObject)
{
	__unsafe_unretained YapDatabaseViewSortingWithObjectBlock sortingBlock =
	    (YapDatabaseViewSortingWithObjectBlock)sorting->block;
	
	YapCollectionKey *another = nil;
	id anotherObject = nil;
	[databaseTransaction getCollectionKey:&another
	                               object:&anotherObject
	                             forRowid:anotherRowid];
	
	return sortingBlock(databaseTransaction, group, 
	                      collectionKey.collection, collectionKey.key,        object,
	                            another.collection,       another.key, anotherObject); // Here the crash happens
}

sortingBlock is the cause, it is called with a nil argument which is declared _Nonnull. According to its declaration it is either object or anotherObject which is nil.

My conclusion The problem is in YapDatabaseViewSortingWithObjectBlock declaration. It actually can return nil for object1 and object2 params but it is not reflected in its declaration. Also it can be a deeper level bug in

[databaseTransaction getCollectionKey:&another
	                               object:&anotherObject
	                             forRowid:anotherRowid];

which returns nil when there's no disk space.

Thus I see here 3 possible solutions:

  1. Change YapDatabaseViewSortingWithObjectBlock declaration and mark object1 and object2 _Nullable
  2. Investigate further and try to handle insufficient disk space in getCollectionKey:object:forRowid:
  3. A crutch - avoid passing sorting blocks from swift and create a shim to handle this.

FYI @flovouin

danillahtin avatar Apr 02 '21 14:04 danillahtin

hi @danillahtin, did any of the solutions help?

jkrukowski avatar Feb 17 '22 09:02 jkrukowski