TuringLang
Feature request: Automatically handle errors by rejecting proposal
It would be useful to have the option to tell tell Turing that if it encounters any errors encountered whatsoever during sampling, to just reject the proposal.
This would make for simpler/more elegant code on the user end. Error handling can be optimised by Turing devs, for everyone's benefit, and relatedly will reduce frustration for new users encountering errors in perfectly well formulated models.
I believe Stan does something like this by default i.e. I have never had to (and wouldn't know how to) handle errors during sampling in Stan. It just throughs a warning, rejects the proposal and keeps going.
Can you explain what errors you have in mind? Would you like Turing to catch any exceptions when calling the model? Or are you thinking about specific numerical issues?
I think usually it's good if your code fails if there is a fundamental issue with it. I also assume that wrapping the model function in a try-catch block would cause slow downs and is e.g. not supported by Zygote. More generally, I don't think it's common to catch errors that generally (and I don't think it's good style), SciML and probably any optimization package will catch at most specific errors.
On the other hand handling specific clearly defined errors, e.g., related to incorrect user input, model specification etc., and presenting users with better error messages is probably a good thing to do (there are some recent improvements in that regard in SciML that are summarized in the latest announcement). We also already handle specific numerical issues in HMC and reject the corresponding proposal (but it could be a bit less verbose, I assume...).
Hi David
From personal experience I sometimes get out of domain errors related to the likelihood function where I know that this should not be possible, given the model specification. In cases like this, a flood of warnings will make it obvious that there is actually something wrong with the specification. But one or two warnings is definitely preferable to the complete loss of a long running chain.
I guess, my initial thought was "Once I'm confident that my model is correct, let me have a setting that will catch (and continue) any exception at all that comes up during sampling". But maybe it would be better if this were restricted to errors during likelihood calculations, since if the user is doing additional calculations that could throw errors then they should be comfortable handling those errors themselves. However, for a fairly vanilla model that has a likelihood function with a restricted domain, it would be nice for (particularly new) users to have an option that warns rather than errors.
Since it's errors in calculating likelihoods, this would also include passing in a vector of the wrong size (e.g. due to failure of ODE solution) to the likelihood function.