Platypus icon indicating copy to clipboard operation
Platypus copied to clipboard

Generating parallel reproducible results

Open taleslimaf opened this issue 4 years ago • 0 comments

To ensure that the seed is assigned during parallelism, I have to change the following class to receive parameters initializer and initargs. That way I can pass a function that instantiates the seed (initializer) and as a parameter which seed I want to use (initargs). This modification does not impact the current code.

class ProcessPoolEvaluator(SubmitEvaluator):
    
    def __init__(self, processes=None, initializer=None, initargs=()):
        try:
            from concurrent.futures import ProcessPoolExecutor
            self.executor = ProcessPoolExecutor(
                max_workers = processes,
                initializer = initializer, 
                initargs = initargs
            )
            super(ProcessPoolEvaluator, self).__init__(self.executor.submit)
            LOGGER.log(logging.INFO, "Started process pool evaluator")
            
            if processes:
                LOGGER.log(logging.INFO, "Using user-defined number of processes: %d", processes)
        except ImportError:
            # prevent error from showing in Eclipse if concurrent.futures not available
            raise
        
    def close(self):
        LOGGER.log(logging.DEBUG, "Closing process pool evaluator")
        self.executor.shutdown()
        LOGGER.log(logging.INFO, "Closed process pool evaluator")

taleslimaf avatar Nov 02 '20 20:11 taleslimaf