realm-kotlin

realm-kotlin copied to clipboard

Currently suspendable write looks like that:
suspend fun <R> write(block: MutableRealm.() -> R): R
Could it be possible to add another write() that would accept suspedable block?
eg:
suspend fun <R> write(block:suspend MutableRealm.() -> R): R

Hi @promanowicz

We actually had a pretty lengthy discussion about this internally. The primary reason for the current behavior is that since transactions are blocking all other write transactions (required to preserve ACID properties), then allowing suspend functions inside the write would mean that you risk blocking other writes while waiting for a network request to complete or something like that. Which would probably not be desirable.

Our current thinking was you could use runBlocking inside the write to make it visible that some expensive operation was running.

We are not against changing the behavior, but would like to understand what kind of method you would like to run that is suspendable?

Hi @cmelchior

Our app performs sync with quite big database. During data fetching and processing we would like to call some suspend functions eg. network calls and do this as part of one write transaction. Our current approach is to use multiple write transactions between fetched chunks of data but this results in 'out of memory' errors on some iphones.

Interesting. Is there a reason you want to do this inside the Realm write transaction? I mean, wouldn't that also work if you fetch the data outside the write transaction and then save it later.

Because your current approach will effectively block any user initiated writes while waiting for the network request to finish.

Ad.1 Thats what we are doing now, but this results in memory issues on some iphones with low ram. From our investigation looks that opening write transactions one by one causes the issue. Ad.2 That's specific case, were user is blocked from using app before full synchronization is finished.

Ok, thank you for the clarification 👍

@cmelchior I have a use case:

I have an abstraction layer for exchangeable databases in my framework Trixnity. Transactions are separated from read and write operations:

interface RepositoryTransactionManager {
    suspend fun <T> writeTransaction(block: suspend () -> T): T
    suspend fun <T> readTransaction(block: suspend () -> T): T
}

class RealmRepositoryTransactionManager(
    private val realm: Realm,
) : RepositoryTransactionManager {
    override suspend fun <T> writeTransaction(block: suspend () -> T): T =
        realm.write {
            runBlocking(RealmWriteTransaction(this)) { // runBlocking is bad
                block()
            }
        }

    override suspend fun <T> readTransaction(block: suspend () -> T): T =
        withContext(RealmReadTransaction(realm)) {
            block()
        }
}

Currently there are implementations for two different SQL Frameworks. One of them (Exposed) uses CoroutineContext.Elements to deliver the transaction to read and write operations. I would like to implement something like this for realm in my application, but this needs the realm write block to be suspendable.

So I need the suspend not to do heavy suspending network operations, but rather to have an elegant way to pass the "transaction" (=MutableRealm) to the actual database implementation of the abstract layer.

Any update @cmelchior ?

Maybe my transaction code could be moved to realm-kotlin?

Right now we need to use runBlocking to run coroutines within realm.write. But runBlocking also means, that all coroutines within the transaction are running on one(!) thread.

It seems like Realm needs it's write modifications to be run on a single thread bound to a transaction. But: it is possible to spawn coroutines and switch back to this single thread for the actual write operations. Similar has been done here with another database framework.

I suggest to introduce a new writeSuspending and readSuspending, which sets a CoroutineContext like here. This would also need something like withRealmRead and withRealmWrite to switch the context to the actual write-thread.

@cmelchior any thoughts?

Hi @benkuly.Thank you for that explanation. I can see in that case that having a suspendable write would be nice. However, we are still a bit reluctant to allow this as the default behavior given the reasons above, also considering you have a workaround.

Having two methods would of course be an option, but it would make the API a bit more confusing.

We will keep this issue open though in hope of gathering feedback from more people.

also considering you have a workaround.

@cmelchior The workaround is really bad because it uses runBlocking and therefore heavy work is sticked to the realm write thread. Maybe the new methods could be hided in some way (e.g. only available as a separate module) or exposed via opt-in to prevent wrong usage.

On the other hand I don't think this is needed. I suggested the use of context switiching as soon as there is an actual realm operation. This would even allow parallel coroutines to write without blocking each others, which seems to be your biggest concern. If you don't trust my referenced example above, here is a more complex, but official one from androidx room: https://github.com/androidx/androidx/blob/cac5ccd68f33cadad31eb0e627fe93911617a5e5/room/room-ktx/src/main/java/androidx/room/RoomDatabaseExt.kt#L54