SpatialConvolution:forward trying to allocate too much memory when cudnn.fastest = true

[mranzinger]

I am working through a problem where a seemingly innocuous configuration is trying to allocate about 21 GiB of GPU memory. The following script will reproduce the error:

require 'cudnn';
cudnn.fastest = true
conv = cudnn.SpatialConvolution(164, 256, 3, 1, 1, 1, 1, 0):cuda()
input = torch.CudaTensor(1, 164, 17, 201)
output = conv:forward(input) -- Crashes here (unless you have that much memory)

The failure seems to stem from the call to cudnnGetConvolutionForwardWorkspaceSize in the createIODescriptors function. I suppose I'm not sure if this is a bug in the code wrapping cuDNN, or if it's cuDNN itself.