mkl_random
mkl_random copied to clipboard
Promote multiprocessing MC runner into mkl_random package
Examples are very useful to comprehend a concept of running mkl_random
using multiprocessing. However, writing a multi-processing orchestration code is labor intensive and error prone.
I am suggesting to design proper API for generalized MC runner with:
- Capability to request independent random states/streams for a given number of workers. Follow NumPy recommended methods under the hood
Possible API:
[streams] = mkl_random.get_independent_streams(n_streams=1, root_seed=None, brng=MT2203, method="seed-sequencing", n_jump_ahead=1)
n_streams
: The number of independent streams
brng
: One of basic RNG available in mkl_random
method
: One of the following "default"
, "seed-sequencing"
, "leapfrog"
, "jump_ahead"
, "key-counter
". The "leapfrog"
method is applicable only for brng
that supports the Leap Frogging technique of sub-sequencing random streams. The "key-counter"
method is applicable for counter-based brng
. Respectively, "jump_ahead" will be applicable to only brng
that supports the Jumping/Skipping Ahead technique. The "seed-sequencing"
will be universally supported by all brng
.
The "default"
method is brng
specific. It will use seed sequencing technique for brng
like MT19937. For MCG59 it will use the "leapfrog"
. For counter-based brng
it will use unique the "key-counter"
technique. The "seed-sequencing"
will be the default method for the rest of brng
.
The function will return the list [stream[0], ..., stream[n_streams-1]
of size n_streams
containing independent streams which can later be accessed by respective process's worker_id
.
- Capability to spawn the required number of processes from
multiprocessing.Pool
and perform a map operation for a user-supplied functionlocal_task(worker_id, stream, *args, **kwargs)
.