Support retrieving resources as flows
Changes
- Added
fun getResourceAsFlow(String, Int): Flow<ByteArray>, which retrieves the resource as a flow of byte array chunks.- On Android & JVM targets, the flow uses
InputStream. - On Apple targets, the flow uses
NSInputStream. - On web targets, the flow uses
ReadableStreamBYOBReader.
- On Android & JVM targets, the flow uses
- Added
ResourceIOException, which wraps internal IO exceptions thrown while reading resources. - Modified the Gradle plugin so the generated
Resclass containsgetAsFlow(String, Int), which invokesgetResourceAsFlow. - Added a section using
Res.getAsFlowandkotlinx.ioin the files tab of the resource demo app. - The default requested size of chunks is 8192, which is the value of
Segment.SIZEofkotlinx.ioorokio.
Motivation
I am making an app that uses deep learning. Although the actual app downloads the DL model from our server, we embed the DL model into our app as a resource for internal testing. I have been handling this by implementing resource reading with platform-specific APIs. I tried refactoring this part using the new resource API, but it didn't work because the app crashed with an OOM error when I used Res.readBytes.
To prevent such an error, I need to read resources as chunks, so I need a proper IO API here. Since kotlinx.io already provides platform-agonistic IO, I tried to implement this with kotlinx.io, but kotlinx.io only provides synchronous IO and is incompatible with web targets, so I failed.
So, I came up with the idea of using Flow for this, as Flow is the only first-party Kotlin API to do something sequentially and asynchronously, as far as I know.
Example
LaunchedEffect(Unit) {
withContext(Dispatchers.IO) {
// copy the resource content to someFileStream
someFileStream.use { stream ->
Res.getAsFlow("files/bigFile.bin").collect { chunk ->
stream.write(chunk);
}
}
}
}