bigstatsr
bigstatsr copied to clipboard
Seems big_univLogReg() is using too many cores.
If using it with ncores = 4
on the cluster, it seems I get 4 R processes (this is how it should be) running each at 400% CPU (this should be only 100%).
Not sure where that comes from, but it would be useful investigating. I recall that, if too much parallelism is used, it can slows things very significantly.
Note: I do have bigparallelr::set_blas_ncores(1)
in my .Rprofile.
It seems I also need RhpcBLASctl::omp_set_num_threads(1)
.
Should I add this to the checks in {bigparallelr]?