abergeron

To see if that is really a problem you can try the new version of the op that is https://github.com/Theano/Theano/pull/2117. If used directly the grad will use the Backward calls....

I re-ran the test with this patch: ``` patch diff --git a/theano/pylearn2_benchmark.py b/theano/pylearn2_benchmark.py index a110628..be4c75d 100644 --- a/theano/pylearn2_benchmark.py +++ b/theano/pylearn2_benchmark.py @@ -203,7 +203,17 @@ for run in runs: mode =...

I've adapted the code to send/receive arrays for some other purpose and got an error from theano about arrays being not aligned. The problem is that if you specify copy=True,...

Do you have a limit on the process? Because that is probably the only thing that could cause a mmap to fail like that. You can check your current limits...

I can't really help anymore then. It works locally and there is no memory leak so the problem comes from the cluster configuration somehow. I will admit that I have...

Is there much to do?

Sorry it took a lot of time, but I think I've addressed the comments now.

I don't know what is wrong with tests/unittests/client/test_runner.py::test_runner_inside_dask, but I'm reasonably certain I didn't touch anything related to it and it keeps failing, but only on python 3.8.

This seems like a convoluted and constructed scenario. Having the minibatch dispatch and the controller in the same process should probably not even be supported since it is super slow...

The minibatch server should not have a limited supply. Or if it is limited it should be enough to fully satisfy each worker. I don't think we should support any...