realm-swift
realm-swift copied to clipboard
realm
Delivering notifications within write transactions makes them hard to use
In v2.1.0 we started delivering collection notifications when beginning a write transaction advances the read version of a Realm. We deliver these notifications within the write transaction to prevent starvation when another thread is frequently writing to the Realm. Users have found that confusing to deal with a notification block potentially being invoked within a write transaction (#4815, #4701, #4511, and likely others). There are certain operations, such as registering for object or collection notifications, that cannot be performed within a write transaction. It's not clear how a user can ever safely use such functionality within a notification callback when it may sometimes be invoked within a write transaction (other than by never performing writes on a thread that is observing a collection or object notification?).
@tgoyne mentioned on Slack that we may be able to deal with this by separating acquisition of the write lock from beginning the write transaction. This would allow us to avoid starvation while also allowing notification callbacks to perform operations that cannot be performed within a write transaction.
Related to this:
the way this mechanism is currently working can cause inconsistencies (crash) in for example a UITableView. If for example two write transactions are performed in the same runloop (or different threads), the second write may trigger the notification synchronously because the realm is advanced to the latest version. In our case we do a tableView.reloadData here. This will cause the tableView to call numberOfRows synchronously, but schedule cellForRow-calls on the next runloop. The second write-transaction causes the results to change (less rows in my testproject for example). This does not trigger the notification before the cellForRow-calls are performed. The tableView still expects more rows than are available in the results now.
I have created a sample app to demonstrate this. See RealmNotifications.zip
The tableview shows the results of Persons that are younger than 18. Initially there is one person in the list. When the crash-button is pressed another Person is added, but in two steps. First it is added with age 0, then in the next write the age is set to 20 (thereby dropping out of the query).
ViewController1: most simple approach. Just a notification-block that calls reloadData. This crashes, because the second write triggers the notification synchronously by advancing.
ViewController2: attempt to workaround this by queuing the notification on the next runloop. This seems to 'solve' the problem.
ViewController3: shows the problem with the workaround of ViewController2. There is still a small gap where there could already have been cellForRows scheduled. This still crashes.
ViewController4: always call refresh after each write-transaction, triggering the notification synchronously.
What is the best way to workaround this problem? I don't have a really good feeling with these two 'workarounds'. The value returned in numberOfRows should always match the available rows (results) in the cellForRow.
There is a mechanism to not notify a specific token, but i do want to be notified actually.
I've been struggling with this for a couple of days. I haven't been able to find a way to prevent table view crashes when applying updates that originate from a realm modified on a background thread. Are there any known workarounds?
Any update on this? I think this should be labeled bug rather than enhancement and given a higher priority because the changes in notifications are unusable. Personally, I've only been able to use reloadData() with a UITableView or UICollectionView. While the examples from @TeunVR are spot on, they don't demonstrate the scenario of receiving updates triggered from changes made on a background thread; I tried applying the hacks he used to get it working to no avail.
I'm not 100% sure this issue is related to #4425 which is exactly what I'm experiencing. If not, I suppose I need to open a new ticket since that one is closed.
I'm running into this exact same problem, with an RxSwift "event pipeline" that flatMaps notifications to new Observables (using RxRealm's Observable.from(..., properties:...)). I have tried everything I know, but cannot think of any way around creating new Observables within an Observable that is called in response to Realm notifications. Any updates on this issue or solutions / workarounds?
Trying to work around this gets into other issues. Using observeOn(
Still don't have any solutions for this issue? Man... almost a year, I still using reloadData() instead of the animation of UITableView.
We have encountered this same issue in our project as well, and our current workaround is to write to the database from separate thread pool threads, so for us this issue is clumsy but not crippling.
It would be nice to have a proper solution within Realm itself however, and as has been pointed out, the fix would seemingly be a rather simple one of acquiring the write lock, delivering the collection notifications, and only then starting the write transaction.
@stuartro I know this thread is quite old, but I'm running into the exact same issue with my RxSwift pipeline. Did you somehow find a workaround?
I have a scenario that Person owns and creates a Dog when Person gets created, if a Person gets added to Realm DB then I observe Dog as I want to know the change of Dog.
So I first observe Person collection, then if a Person get inserted, I immediately create observer to observe Dog(person.dog.observe), then the app crashes with message: "Cannot register notification blocks from within write transactions."
I think realm.beginTransaction(person.insert()) triggers the personCollection.observe block then it calls dog.observe and that's why it has the message above. How should I observe a dog of a person when a person gets inserted/updated?
@raphaklr Yes, I found a solution that works for me. The issue occurred when I tried to perform updates as a result of a notification—which was fired as a result of a preceding update, while the (original) Realm write transaction was still in progress. The solution (at least, what's worked flawlessly for me) is to detect the condition and not start a new write transaction if one is already open). See my post at the link below.
https://github.com/realm/realm-cocoa/issues/4858#issuecomment-411325902
If you want a more detailed code snippet lete know and I'll see what I can do.
Here's what I've done that's almost completely eliminated the problem:
fileprivate var deleteAttempts: Int = 0
/// Deletes the specified conversation permanently.
///
/// - Parameters:
/// - conversationId: ID of conversation to delete
/// - completion: Optional completion with error
func deleteConversation(_ conversationId: String, completion: ((_ error: Error?) -> Void)?) {
let url = "\(baseURL)/v2/me/conversations/\(conversationId)"
deleteAttempts += 1
oauthSession.request(.delete, url, parameters: [:]).responseData { response -> Void in
switch response.result {
case .success(let value):
do {
print(value)
let realm = try Realm()
let conversation = realm.objects(Conversation.self).filter("id == %@", conversationId).first!
if !realm.isInWriteTransaction {
realm.beginWrite()
realm.delete(conversation)
try realm.commitWrite()
self.deleteAttempts = 0
completion?(nil)
} else if self.deleteAttempts <= 10 {
print("Realm in write transaction! Will retry in \(self.deleteAttempts)s...")
Dispatch.delay(Double(self.deleteAttempts), closure: {
self.deleteConversation(conversationId, completion: completion)
})
} else {
print("Too many consecutive failures! Perhaps Realm is stuck in a write transaction?")
self.deleteAttempts = 0
completion?(RLMError.fail as? Error)
}
} catch(let error) {
self.deleteAttempts = 0
completion?(error)
}
case .failure(let error):
self.deleteAttempts = 0
completion?(error)
}
}
}
This retries the write transaction up to 10 times with increasing delays, allowing the current write transaction to complete. When I'm trying to perform multiple transactions at once (such as updating a list of objects), I add a small random modifier to the delay (anywhere between ±0.5 seconds) to avoid continual collisions between updates. So far, I've never seen my app need more than two attempts to complete write transactions with up to ten different objects simultaneously. This way, I can call multiple write transactions from notifications without having to worry about crashing.
I first ran into trouble with notifications in July last year. Our app makes heavy use of applying updates that originate from a realm modified on a background thread and the implementation has not changed since my first comment. After upgrading to Realm 3.11.2 last month, I tried switching from reloadData() back to using change notifications and I have not seen a crash so far.
@tgoyne I think it is safe to say my issue is resolved but I don't know what changed to make that happen.
I just tried the demo project from @TeunVR with Realm 3.13.0 and it still crashes. However, that specific issue can be resolved with by wrapping tableview update calls with DispatchQueue.main.async or with a realm.refresh() call after writes. I think documentation may be needed there; I suppose those new to Realm (or unfamiliar with threading) will footgun it.
I had same issue after removing condition if isInWriteTransaction { return } when it crashes with reason, that thread safe reference cannot be obtained during write transaction and it was caused by unnecessary call to add object to realm, but that object was in realm already.
It has complicated logic, but in general where was some object that holds reference to taken raw pics on disk. that pics ([UIImage]) is appended to array in realm ([String]) with for cycle one by one for each image in write transaction. this appending of filenames notifies realm token and uploading starts when all images are stored as filenames. but after appending filenames I was calling add(object:, update:) because of another logic and that call updates nothing at all and cause write transaction that don't notify realm token with change. So after adding condition to make .add func called only when object isn't in realm already solves the problem.
Been struggling with a notification issue myself. My app sends messages that must be saved in draft form before sending, and then updated after upload to server. The primary key changes so I do a delete+create transaction on a background thread. Both the outgoing and final messages can also be simultaneously modified from main or background threads for other processing.
When these writes are happening quickly, the size of the observed ResultSet can change while another notification is being processed and the table is being batch-updated. Because I use the value of ResultSet.count for the UITableView.numberOfSections() I end up with NSInternalInconsistencyException. (same could apply to numberOfRowsInSection()).
While sitting here dreading the thought of implementing a serial queue for database updates I figured out a workaround. I added var observedSectionCount: Int? and set its value within the .update notification. Now in numberOfSections() I always return observedSectionCount instead of accessing the "live" value from the ResultSet. Race condition appears to be solved. Hope this helps someone.
This would be awesome. I'm trying to create another observer within a notification block but that logic crashes often due to "Cannot register notification blocks from within write transactions."
@IsaiahJTurner it's not ideal but you can do something like this in your notification block:
var create_observer: (() -> Void)!
create_observer = {
if realm.isInWriteTransaction {
DispatchQueue.main.asyncAfter(deadline: .now() + 0.1, execute: {
create_observer()
})
return
}
// your code goes here
}
create_observer()
I use something similar to what @eliburke suggested but with the condition that I try that max 10 times (that should satisfy @shayneoneill) and if it's still in transaction then I let it crash... https://github.com/AckeeCZ/ACKReactiveExtensions/blob/c18ac54fad4ac14241416d8b95096826712bccef/ACKReactiveExtensions/Realm/RealmExtensions.swift#L70
Since I only perform writes on the main thread, I was able to circumvent this issue with this:
DispatchQueue.main.async {
self.notificationToken = thing.observe { change in
// Perform your updates here.
}
}
Anytime I create a notification token, I wrap that in a call to DispatchQueue.main.async. Since writes are synchronous on the thread, the write is always over.
I wrote this nice extension based on previous guys answers. Just use .safelyObserve instead of .observe and you should be good 👍
extension Results where Element: RealmCollectionValue {
/// Same as .observe, but the block is called after write transaction was commited.
func safelyObserve(_ block: @escaping (RealmSwift.RealmCollectionChange<RealmSwift.Results<Element>>) -> Void) -> RealmSwift.NotificationToken {
let dispatchQueue = OperationQueue.current?.underlyingQueue ?? DispatchQueue.main
return self.observe { stuff in
dispatchQueue.async {
block(stuff)
}
}
}
}
@PrzemyslawCholewaDev, How do we know we won't be in a write transaction the next time the block is executed? I was expecting a check for isInWriteTransaction.
