OpenMx icon indicating copy to clipboard operation
OpenMx copied to clipboard

cope with NA values in matrices

Open jpritikin opened this issue 6 years ago • 9 comments

@mcneale wrote:

The propagation of NAs as the result of post-multiplication by a column that includes NAs is a problem that needs to be trapped. Overall I think we should not break backwards compatibility on this one - people have enough trouble getting threshold models to run in the first place. And to converge in the second...

jpritikin avatar Dec 04 '19 15:12 jpritikin

OpenMx has never really supported NAs in matrices. It's too bad the documentation suggested using them. We could automatically replace NAs with some other value. However, I worry that this would preclude a future use of NA.

jpritikin avatar Dec 04 '19 15:12 jpritikin

Wait, we already have a solution? https://openmx.ssri.psu.edu/comment/8538#comment-8538

jpritikin avatar Dec 04 '19 16:12 jpritikin

Yes. There is a specific error message OpenMx gives if one of the thresholds it needs to use is NA.

RMKirkpatrick avatar Dec 04 '19 16:12 RMKirkpatrick

In an algebra, NA's will propagate to elements that are not supposed to be NA. Suppose one ordinal variable, with the lower triangular trick:

y <- matrix(c(1,1,1,0,1,1,0,0,1),3,3) x <- matrix(c(0, 1, NA),3,1) y%*%x [,1] [1,] NA [2,] NA [3,] NA

mcneale avatar Dec 04 '19 16:12 mcneale

@mcneale Correct. And in that case, OpenMx will give the "I hit NA values after only 0 thresholds" error.

RMKirkpatrick avatar Dec 04 '19 16:12 RMKirkpatrick

Ok, so NAs work as directly specified elements, but they don't work if an algebra is being used. That's perhaps as good as it gets, if a bit confusing for the user.

mcneale avatar Dec 04 '19 16:12 mcneale

Ok, so NAs work as directly specified elements, but they don't work if an algebra is being used.

You know what? Maybe at one point, NAs did work if thresholds were specified with an algebra. See issue #240 . We used to use "unsafe" BLAS subroutines that would sometimes evaluate 0 * NA as 0.

RMKirkpatrick avatar Dec 05 '19 21:12 RMKirkpatrick

We used to use "unsafe" BLAS subroutines that would sometimes evaluate 0 * NA as 0.

Ick

jpritikin avatar Dec 05 '19 22:12 jpritikin

Is this resolved? Has anyone appropriately updated the User Guide so that it no longer recommends a broken way to set up thresholds?

RMKirkpatrick avatar Mar 13 '20 18:03 RMKirkpatrick