bigstatsr
bigstatsr copied to clipboard
How to cbind/rbind two big matrices that have the same dimension
Hi, I have two big matrices, both are 30 x 2^29 (30 rows, 2^29 columns), let say bm1 and bm2. I would like to "cbind" the two matrices, i.e. placing the content of bm2 to the right of bm1, which creates a new big matrix that is 30 x 2^30.
I attempted to do that with the following code: #Step 1: Expand bm1 by adding x columns where x is the current number of columns of bm1 bm1$add_columns(bm1$ncol) #Step 2: Place the content of bm2 to the right half of the expanded bm1 bm1[,(bm1$ncol/2 + 1):bm1$ncol] <- bm2[]
I received the "cannot allocate vector of size 60.0 GB" error. Upon further examination, I understand that it is because these two matrices, bm1[,(bm1$ncol/2 + 1):bm1$ncol] and bm2[], which are both 30 x 2^29, can not be called. I also understand that when working with FBM objects, it is generally not a good idea to work the the matrix representations of these objects directly. I have looked at the list of available functions in the bigstatsr package, but I couldn't find one that seems to solve my problem, which is to place the content of a big matrix inside another big matrix.
Could you give me a suggestion on how I can go about solving this problem? Thank you.