Is there any examples on using optunity.parallel.pmap?

[xgdgsc]

I didn' t found any example on it. Can you provide some example?

[claesenm]

This function provides equivalent functionality as Python's map, but uses a threadpool to parallelize evaluations. The usual place to use it in Optunity is within optimize, maximize or minimize, e.g.

import optunity    
f = lambda x, y: x + y
pars, _, _ = optunity.minimize(f, x=[0, 1], y=[0, 1], num_evals=1000, pmap=optunity.pmap)

Note that optunity.pmap == optunity.parallel.pmap.

By default, optunity.parallel.pmap will use the number of cores as the size of the threadpool. If you prefer to set the amount manually, you can use optunity.parallel.create_pmap(threadpool_size), e.g.:

import optunity
f = lambda x, y: x + y
pmap4 = optunity.parallel.create_pmap(4)
pars, _, _ = optunity.minimize(f, x=[0, 1], y=[0, 1], num_evals=1000, pmap=pmap4)

I just noticed that optunity.parallel.create_pmap isn't properly documented yet, I'll do that ASAP.

[xgdgsc]

Thanks. optunity.parallel.create_pmap is what I want.

[ab-trader]

This was returned by the example shown above in Python 2.7.10:

Traceback (most recent call last): File "", line 1, in File "D:\Python27\lib\site-packages\optunity\api.py", line 212, in minimize pmap=pmap) File "D:\Python27\lib\site-packages\optunity\api.py", line 245, in optimize solution, report = solver.optimize(f, maximize, pmap=pmap) File "D:\Python27\lib\site-packages\optunity\solvers\ParticleSwarm.py", line 270, in optimize fitnesses = pmap(evaluate, list(map(self.particle2dict, pop))) File "D:\Python27\lib\site-packages\optunity\parallel.py", line 78, in pmap p.start() File "D:\Python27\lib\multiprocessing\process.py", line 130, in start self._popen = Popen(self) File "D:\Python27\lib\multiprocessing\forking.py", line 277, in init dump(process_obj, to_child, HIGHEST_PROTOCOL) File "D:\Python27\lib\multiprocessing\forking.py", line 199, in dump ForkingPickler(file, protocol).dump(obj) File "D:\Python27\lib\pickle.py", line 224, in dump self.save(obj) File "D:\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "D:\Python27\lib\pickle.py", line 425, in save_reduce save(state) File "D:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "D:\Python27\lib\pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "D:\Python27\lib\pickle.py", line 687, in _batch_setitems save(v) File "D:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "D:\Python27\lib\pickle.py", line 554, in save_tuple save(element) File "D:\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "D:\Python27\lib\pickle.py", line 754, in save_global (obj, module, name)) pickle.PicklingError: Can't pickle <function at 0x03F1E230>: it's not found as main.

Traceback (most recent call last): File "", line 1, in File "D:\Python27\lib\multiprocessing\forking.py", line 381, in main self = load(from_parent) File "D:\Python27\lib\pickle.py", line 1384, in load return Unpickler(file).load() File "D:\Python27\lib\pickle.py", line 864, in load dispatchkey File "D:\Python27\lib\pickle.py", line 886, in load_eof raise EOFError EOFError

Could you please advice what need to be done to run optunity parallel?

[gunju]

Yes. Even I am getting a similar kind of error. I tried to run it in python console. Still getting the same error.

This works well in Ubuntu. However, if I run the above code, then without using 'pmap' gives faster output, compared to with 'pmap'.