bigstatsr icon indicating copy to clipboard operation
bigstatsr copied to clipboard

Seems big_univLogReg() is using too many cores.

Open privefl opened this issue 3 years ago • 1 comments

If using it with ncores = 4 on the cluster, it seems I get 4 R processes (this is how it should be) running each at 400% CPU (this should be only 100%).

Not sure where that comes from, but it would be useful investigating. I recall that, if too much parallelism is used, it can slows things very significantly.

Note: I do have bigparallelr::set_blas_ncores(1) in my .Rprofile.

privefl avatar May 26 '21 12:05 privefl

It seems I also need RhpcBLASctl::omp_set_num_threads(1).

Should I add this to the checks in {bigparallelr]?

privefl avatar Jun 10 '21 07:06 privefl