mkl_random icon indicating copy to clipboard operation
mkl_random copied to clipboard

Promote multiprocessing MC runner into mkl_random package

Open samaid opened this issue 1 year ago • 0 comments

Examples are very useful to comprehend a concept of running mkl_random using multiprocessing. However, writing a multi-processing orchestration code is labor intensive and error prone.

I am suggesting to design proper API for generalized MC runner with:

  • Capability to request independent random states/streams for a given number of workers. Follow NumPy recommended methods under the hood

Possible API:

[streams] = mkl_random.get_independent_streams(n_streams=1, root_seed=None, brng=MT2203, method="seed-sequencing", n_jump_ahead=1)

n_streams: The number of independent streams brng: One of basic RNG available in mkl_random method: One of the following "default", "seed-sequencing", "leapfrog", "jump_ahead", "key-counter". The "leapfrog" method is applicable only for brng that supports the Leap Frogging technique of sub-sequencing random streams. The "key-counter" method is applicable for counter-based brng. Respectively, "jump_ahead" will be applicable to only brng that supports the Jumping/Skipping Ahead technique. The "seed-sequencing" will be universally supported by all brng.

The "default" method is brng specific. It will use seed sequencing technique for brng like MT19937. For MCG59 it will use the "leapfrog". For counter-based brng it will use unique the "key-counter" technique. The "seed-sequencing" will be the default method for the rest of brng.

The function will return the list [stream[0], ..., stream[n_streams-1] of size n_streams containing independent streams which can later be accessed by respective process's worker_id.

  • Capability to spawn the required number of processes from multiprocessing.Pool and perform a map operation for a user-supplied function local_task(worker_id, stream, *args, **kwargs).

samaid avatar Mar 22 '23 18:03 samaid