NullPointerException in Query.executeAsList

[savvasdalkitsis]

SQLDelight Version

1.5.5

Application Operating System

Android

Describe the Bug

This seems like a dupe of https://github.com/cashapp/sqldelight/issues/1584 but on the latest 1.x version of the library.

I keep getting these locally in some situations and from from my beta users.

Stacktrace

Failed to read row 3345, column 0 from a window with 3344 rows, 8 columns
Failed to read row 3345, column 0 from a window with 3344 rows, 8 columns

java.lang.NullPointerException
	at com.savvasdalkitsis.uhuruphotos.feature.db.domain.api.api.RemoteMediaCollectionsQueriesImpl$getRemoteMediaCollections$1.invoke(DatabaseImpl.kt:1489)
	at com.savvasdalkitsis.uhuruphotos.feature.db.domain.api.api.RemoteMediaCollectionsQueriesImpl$getRemoteMediaCollections$1.invoke(DatabaseImpl.kt:1487)
	at com.squareup.sqldelight.Query.executeAsList(Query.kt:135)
	at com.squareup.sqldelight.runtime.coroutines.FlowQuery$mapToList$1$1.invokeSuspend(FlowExtensions.kt:98)
	at com.squareup.sqldelight.runtime.coroutines.FlowQuery$mapToList$1$1.invoke(Unknown Source:8)
	at com.squareup.sqldelight.runtime.coroutines.FlowQuery$mapToList$1$1.invoke(Unknown Source:4)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:167)
	at kotlinx.coroutines.BuildersKt.withContext(Unknown Source:1)
	at com.squareup.sqldelight.runtime.coroutines.FlowQuery$mapToList$$inlined$map$1$2.emit(Collect.kt:137)
	at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:15)
	at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:15)
	at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:87)
	at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:66)
	at com.squareup.sqldelight.runtime.coroutines.FlowQuery$asFlow$1.invokeSuspend(FlowExtensions.kt:49)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
	Suppressed: java.lang.NullPointerException
		... 21 more
	Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@3380e17, Dispatchers.Main.immediate]

[AlecKazakova]

yea this is usually a sign that some of the data being put into SQLite is too big and needs to be stored elsewhere on disk then a URI is stored in SQLite instead.

SQLite itself can handle it, but the Android bridge ends up crashing on read

[AlecKazakova]

ideally if theres a way to get one of these DBs thats failing on read you could inspect and try to find out whats in that row causing it to fail

[savvasdalkitsis]

Would the size issue apply to the whole cursor data or on a per row basis? cause the query that crashes it for me returns very little data per row. This is an example row:

36151 | 17 |   | Shepherd's Bush | 0123825fc861c820a752375eecfdedf01 | #1e1d23 | 0 | 1.5 | image

[AlecKazakova]

actually now that I'm re-reading the stack trace it looks like its trying to read a row beyond the cursor row length...interesting. Not sure what would cause that

[savvasdalkitsis]

I have a somewhat reproducible usecase in my local environment but it's not 100%. has to do something with the data i have loaded and my specific configuration.

Happy to either provide any more logging/info you might need to help pin it down or even join a debugging session with you if you are so inclined :)

[andrewfarm]

We have users encountering what appears to be the same issue. I'm unable to reproduce it locally, but the error reports indicate that in each case, the cursor is trying to read beyond the number of rows that actually exist in the result set. Also, it has only occurred when there are thousands of rows in the result set. The rows do not contain any BLOBs. As a workaround, immediately retrying the query usually succeeds.

[JsFlo]

Sharing that I was also running into this and was able to get a small repro so that hopefully we get a solution for this: https://github.com/sqldelight/sqldelight/issues/6049

I've seen workarounds and I'll share another workaround (not advocating for it though) and that's to make your mappers return nullable columns. You don't need to change the column in the schema but you can massage the select to make the generated mappers return non-null and this lets you handle this case in the application level. This is hacky and I'm hoping with a easily reproducible case I shared, we can find a better/real solution