mice icon indicating copy to clipboard operation
mice copied to clipboard

Not able to reproduce previous imputation with identical code and data

Open martelief opened this issue 2 years ago • 2 comments

Hi,

I have performed an imputation several weeks ago that I am now wanting to reproduce, but I keep getting different results and I do not understand why. I am using the same underlying data, the same predictor matrix, the same imputation methods and the same seed. The imputation itself is functioning fine and giving results that I would expect, so no problems there.

My previous imputation was done in R 4.0.5 and mice 3.13.0 whilst my latest imputation was in the most recent version of R and the mice package. So, thinking this must be the issue, I tried imputing the data in both the old R version and in the most recent R version, but I get the exact same results for both imputations. However, they are not the same as the previous imputation that I did.

The only thing I can seem to find is that the number of loggedEvents (but not the type/contents of the loggedEvents) differ slightly for the old imputation and the new one. And the lastSeedValues seem to be different for both imputations (except for the first one that is listed). I think this might be indicative of a problem, but I am not sure of what the lastSeedValues mean and what they are used for, and whether I can override them to reproduce my imputation. Could someone perhaps explain this and help me out?

martelief avatar Jun 23 '22 11:06 martelief

PR #432 changed the random seed mechanism. For exact reproducibility, you may need to install a version prior to mice 3.13.11.

stefvanbuuren avatar Jun 24 '22 14:06 stefvanbuuren

Another solution might be to use set.seed() in the global environment, outside the mice() call, which is, I think, basically what the mice version prior to 3.13.11 did.

thomvolker avatar Jun 24 '22 14:06 thomvolker

Likely to be related to changes in seed. In general, should not affect interpretation of the results.

We cannot help without a reprex so closing.

stefvanbuuren avatar Nov 13 '22 16:11 stefvanbuuren