vertx-sql-client
vertx-sql-client copied to clipboard
eclipse-vertx
Application not shutting down/crashing on Exception
I have a reactive pipeline that processes some data and then at some point stores as well as reads data from a psql database. I have noticed now if one part of the applciation thows an exception the whole pipeline stops bot the application does not crash as I would have expected. If I take out the psql interaction but leave everything as-is the application crashes as I expected. Even when I switch out the psql interaction with another client such as https://github.com/jasync-sql/jasync-sql the application works and crashes as I would expect.
My vertx-sql-client usage is quite simple:
initialisation:
...
private val connectOptions = pgConnectOptionsOf(
port = host.substring(host.indexOf(":") + 1).toInt(),
host = host.substring(0, host.indexOf(":")),
database = database,
user = username,
password = password
)
private val client = PgPool.pool(connectOptions, poolOptionsOf())
....
the actual querying:
...
client.preparedQueryAwait(query, value).map {
emit(getRowEventMapper().invoke(it))
}
...
The emit is just emitting the data into a Kotlin Flow stream. My guess is that it has something to do with the vertx event-loop, but I am really not sure what exactly keeps my application from crashing and what I can do that it does crash. Any ideas?
Ok, I found the culprit, if I catch all errors from the pipeline, then run client.close() and rethrow the error the application crashes as expected. While I am ok to add this workaround how come this for now is there a way to fix this in the client directly?
Yep, the application is basically this:
someFlowProducer
.map(::someComputation)
.map(dbStore::process)
.map(computation::process)
.catch { exception ->
dbStore.shutdown()
throw exception
}
.collect()
Where-as dbStore is the class that holds the pgpool and is responsible for the inserts and selects. The process method inserts elements and in the computation class we do a select and some further computation. The catch operator is described here: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/catch.html and dbStore.shutdown() literally just calls client.close()
To be a bit more specific, the issue I ran into was within the someFlowProducer I was using an s3 client to continuously fetch data from a bucket and it was throwing the unexpected exception which did not crash the whole application but brought the flow to halt.
Hope that makes more sense. Let me know if I can provide any more information.
what would you expect to work instead ?
what does dbStore.shutdown() do ?
what is the equivalent snippet that works with jasyncql ?
I'm assuming Kotlin Flow stream is https://kotlinlang.org/docs/reference/coroutines/flow.html ?
Yes, I am using https://kotlinlang.org/docs/reference/coroutines/flow.html. As I said, dbStore.shutdown() just calls client.close() - I guess at the very least I am surprised about the different behaviour between jasync-sql and vertx-sql-client. How come the vertx-sql-client can block the application from shutting down on an exception?
And yes if there is a solution to fix this then vertx-lang-kotlin would also be a good place for this.
"How come the vertx-sql-client can block the application from shutting down on an exception?"
can you elaborate ? I am not familliar with Kotlin Flow and I would like to understand better what is the actual issue with the SQL client.
Ok I will put up a full reproduction case, to illustrate exactly what I am talking about.
thanks.
On 17 Oct 2019, at 11:56, Markus Padourek [email protected] wrote:
Ok I will put up a full reproduction case, to illustrate exactly what I am talking about.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/eclipse-vertx/vertx-sql-client/issues/435?email_source=notifications&email_token=AABXDCT2QRG65W7NVLONHNLQPAZFBA5CNFSM4JBKS3N2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBPQ3WI#issuecomment-543100377, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABXDCXTZZXQGQV5PGFTPSLQPAZFBANCNFSM4JBKS3NQ.
