drewmccormack
Ensembles 1.7 multiple problems and disabled functionality on Xcode 9
Ensembles 1.7 is unable to load its omo on Xcode 9, resulting in completely disabled functionality. Firstly it gives the error:
CoreData: annotation: Failed to load optimized model at path '/var/containers/Bundle/Application/46C61E42-DA78-460D-82B1-2B780511C9D8/MyApp.app/CDEEventStoreModel.momd/CDEEventStoreModel_2.omo'
And then we see
CoreData: error: -addPersistentStoreWithType:Binary configuration:(null) URL:file:///var/mobile/Containers/Data/Application/FBE36631-CF92-420E-A4D3-5EBA07B2447B/Library/Application%20Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/transitcache/MyAppSync/download/236_159945E3-FA38-46B2-9BC3-394C2B07FD10-960-00000081E9732578_A23EF461.cdeevent options:{ NSInferMappingModelAutomaticallyOption = 1; NSMigratePersistentStoresAutomaticallyOption = 1; } ... returned error Error Domain=NSCocoaErrorDomain Code=259 "The file couldn’t be opened because it isn’t in the correct format." UserInfo={NSUnderlyingException=Can't read binary data from file, NSUnderlyingError=0x1c0647440 {Error Domain=NSCocoaErrorDomain Code=259 "The file “236_159945E3-FA38-46B2-9BC3-394C2B07FD10-960-00000081E9732578_A23EF461.cdeevent” couldn’t be opened because it isn’t in the correct format." UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/FBE36631-CF92-420E-A4D3-5EBA07B2447B/Library/Application Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/transitcache/MyAppSync/download/236_159945E3-FA38-46B2-9BC3-394C2B07FD10-960-00000081E9732578_A23EF461.cdeevent, NSUnderlyingException=value for key 'NS.objects' was of unexpected class 'CDEPropertyChangeValue'. Allowed classes are '{( NSNumber, NSUUID, NSOrderedSet, NSDate, NSSet, NSDictionary, NSNull, NSDictionaryMapNode, NSData, NSArray, NSString, NSDecimalNumber, NSURL )}'.}}} with userInfo dictionary { NSUnderlyingError = "Error Domain=NSCocoaErrorDomain Code=259 \"The file \U201c236_159945E3-FA38-46B2-9BC3-394C2B07FD10-960-00000081E9732578_A23EF461.cdeevent\U201d couldn\U2019t be opened because it isn\U2019t in the correct format.\" UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/FBE36631-CF92-420E-A4D3-5EBA07B2447B/Library/Application Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/transitcache/MyAppSync/download/236_159945E3-FA38-46B2-9BC3-394C2B07FD10-960-00000081E9732578_A23EF461.cdeevent, NSUnderlyingException=value for key 'NS.objects' was of unexpected class 'CDEPropertyChangeValue'. Allowed classes are '{(\n NSNumber,\n NSUUID,\n NSOrderedSet,\n NSDate,\n NSSet,\n NSDictionary,\n NSNull,\n NSDictionaryMapNode,\n NSData,\n NSArray,\n NSString,\n NSDecimalNumber,\n NSURL\n)}'.}"; NSUnderlyingException = "Can't read binary data from file"; }
I am not sure how I can proceed here, so I wanted to open an issue for this.
How are you linking in Ensembles? Are you building yourself in your project? Cocoapods? Precompiled binaries?
Cocoapods. Completed the handbook and it has been working until Xcode 9. I think it has to do something with CDEPropertyChangeValue being an NSObject instead of the new allowed classes. So it cannot read the database which was created using Xcode 8 compiled app. With a brand new app that does not have any remnant iCloud files, I think the error was different. I'll need to test that again. The failing of optimized model is always there though.
Checked it now, when I am using an app that is compiled in Xcode 9 and has not done any syncing before, the app fails to load the model file and after that it cannot do anything else related to Ensembles.
It is pretty clear the problem is loading the model file. Without that, it will not know about CDEPropertyChangeValue class, giving the other errors.
I guess a solution may be for me to add a new version compiled with Xcode 9. You could perhaps test if this will work by building Ensembles separate to your project, grabbing the CDEEventStoreModel.momd out of it, and replacing the Cocoapods one with that.
Drew
I must say, it seems odd. I would think the file format would be back compatible. Are you sure it isn't just a glitch with the file in Cocoapods? Does it help to clean the caches and completely install again?
Well, I tried everything. I think it is not about Cocoapods. I encountered this after creating a new app with Xcode 9, so there were no caches. Then I tried to see if this was about that app only and then I saw that my earlier app was having problems when compiled on Xcode 9. When I switch to Xcode 8, everything works fine.
Have you tried just creating an app without Cocoapods, dragging the Ensembles Xcode project in, and linking that way? Still a problem?
I am currently trying exactly that. I need to drag Ensembles iOS project to the workspace and link the framework, right?
Yep. See README.
On 15 Aug 2017, at 10:29, keremerkan [email protected] wrote:
I am currently trying exactly that. I need to drag Ensembles iOS project to the workspace and link the framework, right?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/drewmccormack/ensembles/issues/259#issuecomment-322410338, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEuAOfXnZ_ocu6T0jHgftg315iVfTkoks5sYVbqgaJpZM4O3ScG.
Can now load the model file, but the other problem still exists. It looks like the sync file is created but as CDEPropertyChangeValueis of type NSObject, SQLite rejects reading it. This seems to be over my head currently :)
The CDEPropertyChangeValue uses NSCoding. SQL should never see it. Core Data should be transforming those change values into data, and storing that.
Is your app in Swift? Just wondering if there is some problem with it is looking in the wrong place for the module.
Could the problem be due to old data in the cloud? Maybe that is now corrupted. Be worth clearing the cloud data.
This is still with the pristine app. Removed everything in the cloud and compiled and ran the app. I am using Objective-C.
Here is the error when I enable syncing and add a record on the other device:
-[CDEEventMigrator migrateEventsInFromFiles:completion:]_block_invoke line 180: Failed to migrate modification events: Error Domain=NSCocoaErrorDomain Code=259 "The file couldn’t be opened because it isn’t in the correct format." UserInfo={NSUnderlyingException=Can't read binary data from file, NSUnderlyingError=0x1c4247410 {Error Domain=NSCocoaErrorDomain Code=259 "The file “0_A3FFF4B8-703D-4D2B-992B-FEB59F6CE43B-401-000000196B8033A8_5FB34B05.cdeevent” couldn’t be opened because it isn’t in the correct format." UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/A9082474-E266-4F26-A3BB-EC0756DA454A/Library/Application Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/transitcache/ListsSync/download/0_A3FFF4B8-703D-4D2B-992B-FEB59F6CE43B-401-000000196B8033A8_5FB34B05.cdeevent, NSUnderlyingException=value for key 'NS.objects' was of unexpected class 'CDEPropertyChangeValue'. Allowed classes are '{( NSDecimalNumber, NSURL, NSNumber, NSUUID, NSOrderedSet, NSDate, NSSet, NSDictionary, NSNull, NSDictionaryMapNode, NSData, NSArray, NSString )}'.}}}
And when I deleech, there are many errors on I/O with the events.sqlite It may be a bug in iOS 11 SQLite implementation, I am not sure.
Seems as if for some reason either the proper keyed-archive transformation is not working (you could check the CDEEventStore model and see if that looks OK), or for some reason the CDEPropertyChangeValue class is not properly loaded from the library, which would be odd.
I will try to test this myself in the coming days and see what I can find.
FWIW I have reproduced this problem using the Idiomatic sample. I suspect a bug in Core Data's binary store code when using transformables. I will investigate this further and see if I can make a simple example exhibiting the problem for Apple.
Note that Ensembles 2 does not use the Core Data binary format, it uses JSON instead, so this issue only affects Ensembles version 1.x in conjunction with Xcode 9.
I hope that it can be fixed. I love Ensembles and hope to migrate to Ensembles 2 soon, but currently it is not feasible.
It seems to definitely be a Core Data bug. I have put in radar 33900451. An example of the problem is in this project (which doesn't even include Ensembles).
https://www.dropbox.com/s/htg0m0fq4asqz8f/BinaryStoreWithTransformableProperty.zip?dl=0
If you run that app once to create a store, then kill the app, and run it again, you get the same error. Seems to be that transformable properties aren't working with binary stores.
Ensembles 2 will work. You can try it out for free by taking a subscription, and cancelling before the first month is up.
OK, got clarification from Apple. It is a very fresh change to the Core Data APIs. I will need to add metadata to workaround it.
/*
Allows developers to provide an additional set of classes (which must implement NSSecureCoding) that should be used while
decoding a binary store.
Using this option is preferable to using NSBinaryStoreInsecureDecodingCompatibilityOption.
*/
COREDATA_EXTERN NSString * const NSBinaryStoreSecureDecodingClasses API_AVAILABLE(macosx(10.13),ios(11.0),tvos(11.0),watchos(4.0));
/*
Indicate that the binary store should be decoded insecurely. This may be necessary if a store has metadata or transformable
properties containing non-standard classes. If possible, developers should use the NSBinaryStoreSecureDecodingClasses option
to specify the contained classes, allowing the binary store to to be securely decoded.
Applications linked before the availability date will default to using this option.
*/
COREDATA_EXTERN NSString * const NSBinaryStoreInsecureDecodingCompatibilityOption API_AVAILABLE(macosx(10.13),ios(11.0),tvos(11.0),watchos(4.0));
Will try to do this in coming days. If you are in a hurry, you could try adding the second one to the metadata where the binary store is imported.
Well, hit the "Undefined symbols" error. It should be defined in Core Data but looks like it is not, yet. Or I am doing something wrong.
Undefined symbols for architecture arm64: "_NSBinaryStoreInsecureDecodingCompatibilityOption", referenced from: ___51-[CDEPersistentStoreImporter importWithCompletion:]_block_invoke in CDEPersistentStoreImporter.o
I was told they only added it on the beta yesterday (6?)
Kind regards, Drew
On 15 Aug 2017, at 22:24, keremerkan [email protected] wrote:
Well, hit the "Undefined symbols" error. It should be defined in Core Data but looks like it is not, yet. Or I am doing something wrong.
Undefined symbols for architecture arm64: "_NSBinaryStoreInsecureDecodingCompatibilityOption", referenced from: ___51-[CDEPersistentStoreImporter importWithCompletion:]_block_invoke in CDEPersistentStoreImporter.o
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I found that it was added on beta 5. Xcode recognises and autofills the option, but fails when building. And it does not yet have a beta 6 (even though iOS has one), so I think we will have to wait.
I was told it is only for apps targeting iOS 11, so maybe just targeting iOS 10 may avoid it.
On 15 Aug 2017, at 22:32, keremerkan [email protected] wrote:
I found that it was added on beta 5. Xcode recognises and autofills the option, but fails when building. And it does not yet have a beta 6 (even though iOS has one), so I think we will have to wait.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Unfortunately, this app has to target iOS 11. And the older one already targets iOS 9 and later, maybe I can be able to use Xcode 8 for that one for some time until this is fixed. But I am unable to use syncing for the new app for the time being. It cannot be released at the time being anyway as I'll need to wait for iOS 11 to be released. I hope everything will work correctly by then. :)
I was told by the Core Data manager that this functionality was already in the beta, but very recent. I will try it, and if it isn’t, will get back to him about it.
Drew
Apple says the SDK is not up to date, but the system is, or something like that. In other words, the option should work, but you can't use the variable. Apparently we can just use the string
"_NSBinaryStoreInsecureDecodingCompatibilityOption"
I have just tested my little test app with this string as key, and indeed it works. I will push it to the master of GitHub, so you could use that with Cocoapods directly (won't make a release just yet).
Yep, just tested this in Idiomatic, and seems to solve the issue. Have pushed to master.
Yes, it is fixed. Now I need to find why I get libsqlite3 errors when deleeching :) I think this may indeed be a bug in iOS.
[logging] BUG IN CLIENT OF libsqlite3.dylib: database integrity compromised by API violation: vnode unlinked while in use: /private/var/mobile/Containers/Data/Application/9D5FA7A6-65CF-4BC5-98BF-772CB6A93C92/Library/Application Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/ListsSync/events.sqlite [logging] invalidated open fd: 9 (0x11) [logging] BUG IN CLIENT OF libsqlite3.dylib: database integrity compromised by API violation: vnode unlinked while in use: /private/var/mobile/Containers/Data/Application/9D5FA7A6-65CF-4BC5-98BF-772CB6A93C92/Library/Application Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/ListsSync/events.sqlite-shm [logging] invalidated open fd: 11 (0x11) [logging] BUG IN CLIENT OF libsqlite3.dylib: database integrity compromised by API violation: vnode unlinked while in use: /private/var/mobile/Containers/Data/Application/9D5FA7A6-65CF-4BC5-98BF-772CB6A93C92/Library/Application Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/ListsSync/events.sqlite-wal [logging] invalidated open fd: 10 (0x11) [error] error: (6922) I/O error for database at /var/mobile/Containers/Data/Application/9D5FA7A6-65CF-4BC5-98BF-772CB6A93C92/Library/Application Support/com.myapp.MyApp/com.mentalfaculty.ensembles.eventdata/ListsSync/events.sqlite. SQLite error code:6922, 'disk I/O error'
Yes, I saw those errors too. I think it is because deleeching deletes the sqlite store that is used internally by Ensembles. Should be harmless, because you don't need that anymore, but if the message persists, I will look into it more.
@drewmccormack I just added pull request #260 to fix this issue. The code works correctly on both iOS 11 and iOS 10. Apps on both iOS versions can now sync correctly with each other.