CUDNN_STATUS_BAD_PARAM when trying to perform forward pass

[tryckfel]

trafficstars

Hi, I'm interested in testing the speed and memory-consumption of your network, and so I downloaded the files linked to in the 'Training on TINY SET' section. I downloaded branch 0.15.5 of NVIDIA/caffe, and compiled it with CuDNN v.4 enabled. So far, so good. But, when I try the benchmark (caffe time) on deploy.prototxt, using the GPU, I get the following error:

Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM *** Check failure stack trace: *** @ 0x7fc1a5c9e778 (unknown) @ 0x7fc1a5c9e6b2 (unknown) @ 0x7fc1a5c9e0b4 (unknown) @ 0x7fc1a5ca1055 (unknown) @ 0x7fc1ab5d39bd caffe::CuDNNConvolutionLayer<>::Forward_gpu() @ 0x4099c2 time() @ 0x405ca3 main @ 0x7fc19e56bb45 (unknown) @ 0x40644c (unknown) @ (nil) (unknown)

Running it on the CPU works. Do you have any idea what could be causing this? Thanks!

[smichalowski]

I've used branch v0.15.5 and CUDNN 5

[tryckfel]

Thanks @smichalowski now that I switched to CuDNN 5 the benchmark works. However, when I try to do a classification with build/cpp_classification/classification.bin, using the snapshot_iter_11704.caffemodel, I get the following error:

F0722 15:50:58.228097 31925 gpu_memory.cpp:68] Check failed: initialized_ Create GPUMemory::Scope to initialize Memory Manager *** Check failure stack trace: *** @ 0x7f798bfc1778 (unknown) @ 0x7f798bfc16b2 (unknown) @ 0x7f798bfc10b4 (unknown) @ 0x7f798bfc4055 (unknown) @ 0x7f799174702a caffe::GPUMemory::Manager::try_allocate() @ 0x7f7991812c53 caffe::GPUMemory::MultiWorkspace::reserve() @ 0x7f79918178df caffe::CuDNNConvolutionLayer<>::Reshape() @ 0x7f79918a7785 caffe::Net<>::Init() @ 0x7f79918a948c caffe::Net<>::Net() @ 0x406b65 Classifier::Classifier() @ 0x4042ce main @ 0x7f7984a96b45 (unknown) @ 0x404830 (unknown) @ (nil) (unknown)

Have you by any chance experienced this error before?