Non-contiguous sub.big.matrix?

[cdeterman]

Currently we have sub.big.matrix but it explicitly states that non-contiguous subsets are not possible. It would be great if we could find a way to get this implemented before the next release.

[kaneplusplus]

Agreed. The sub.big.matrix was implemented with row and column offsets in the BigMatrix class. We could probably do a lot more by implementing a new accessor class. However, we need to make sure that the data structure doing the mapping from matrix coordinate to memory location is not so complex that it uses a lot of memory and we need some way to make it clear that it won't work with bigalgebra.

[cdeterman]

@kaneplusplus why is it that it won't work with bigalgebra? That seems like a pretty large constraint if you want to do anything with the data other than some minimal summary statistics.

[kaneplusplus]

bigalgebra assumes BLAS/LAPACK and both take contiguous column-major c-arrays as input. If your sub.big.matrix is simply a subset of the columns then there is no problem since the memory is contiguous. Otherwise you end up memory that is not contiguous.

A quick fix would be to have bigalgebra check to see if the object is a sub.big.matrix create an anonymous big.matrix and pass that off to the linear algebra routines.

[privefl]

EDIT : Proposition for deepcopy already implemented.

[kaneplusplus]

Thanks very much!

Please note that deepcopy does let you specify columns and rows to copy from a big.matrix object.

[cdeterman]

@privefl Thanks for jumping in, contributions are always welcome. However as noted by Mike this is available via deepcopy. The reason I created this issue was it would be ideal to have a method to reference non-contiguous elements within a big.matrix object WITHOUT copying the elements to another object.

[privefl]

I'm very sorry, I read the documentation too quickly, I thought rows and cols stood for sizes, not indices. I will try this option right now. Thank you both.