JavaGateway attempting to pop from an empty queue

[LuxMiranda]

I'm attempting to run EMD on many threads, and I am encountering this error after it has been running for some time:

Traceback (most recent call last):
  File "/home/lux/miniconda3/envs/emd/lib/python3.6/site-packages/py4j/java_gateway.py", line 982, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

Some preliminary googling seems to reveal that this might be an out-of-memory issue, possibly remedied by increasing the amount of available memory via passing a Java parameter per this comment. Though, this may of course happen anyway if the memory allotment is already maxed out for one's hardware. In this case it might be helpful to have the error be caught in NL4Py/EMD to display a more useful error message.

I realize that this may be an issue with NL4Py, but I'm positing here for visibility since I'm unsure if this error would be thrown in another application, and EMD is the primary usecase for NL4Py at the moment as far as I'm aware :)

[chathika]

@LuxMiranda Thanks for catching this. Yes, that does sound like an issue for NL4Py. Could you provide more details to help replicate the problem, please? How many cores does your system have and how many generations were run successfully before crashing? Also, hope you're using v2.0.0 :)

[LuxMiranda]

@chathika I am using v2.0.0! :) Thank you for fixing the parallelization!

I was running into this issue using as few as 10 cores on my 16-core machine. It's got 16 GB of RAM, as well. I'm running EMD with 43 possible factors using the following parameters:

emd.set_population_size(10)
emd.set_mutation_rate(0.3)
emd.set_crossover_rate(0.8)
emd.set_generations(50)
emd.set_depth(2,8)

It was able to get about 15-20 generations in before throwing the error. Using fewer cores definitely seems to solve the issue.