rfunctions icon indicating copy to clipboard operation
rfunctions copied to clipboard

Overflow in colsums

Open privefl opened this issue 9 years ago • 0 comments

Hello,

I personally use a big.matrix of 0, 1, 2 only. So I use type char. If I make no mistake, the maximum char is 127.

So when using this function on a big.matrix (so, with a large number of rows)

Map<MatrixXchar> bM = Map<MatrixXchar>((char *)xpMat->matrix(), xpMat->nrow(), xpMat->ncol());
Vectorchar colSums = bM.colwise().sum();

it will results in an overflow (the results will be between -128 and 127 whereas it should be more than 10,000 maybe).

Would it be possible (and more pertinent) to replace Vectorchar colSums by VectorXi colSums for char and short types?

Anyway, thanks for introducing me to Mapping big.matrix objects in Eigen.

Florian

privefl avatar Jul 17 '16 06:07 privefl