CUDA driver verision is insufficient for CUDA runtime version

[sghilardi]

Hello, I am trying to set up the U-Net server backend using the simple docker method (startServer.sh). I can successfully create the docker container and connect to it using the U-Net Fiji plug-in, but during segmentation I get the following error:

Model check failed: F0310 19:40:56.398861 55 common.cpp:152] Check failed: error == cudaSuccess (35 vs. 0) CUDA driver version is insufficient for CUDA runtime version

Does anyone have any insight into how to fix the driver/runtime error in Docker? Here is the full Fiji log for the event: (note that after the error message, I closed the docker window which is why there are 'Host Unreachable' errors at the end.)

Establishing SSH connection for '[email protected]:22'
[email protected]$ mkdir ~/data1; mkdir ~/cellnet;
[email protected] $ mkdir "/home/unetuser/~"
[email protected] $ mkdir "/home/unetuser/~/cellnet"
$ sftp "/tmp/unet-12ffe596-e444-4995-b0a3-7a59477999d87163029522390179009.tmp" "[email protected]:22:/home/unetuser/~/cellnet/unet-12ffe596-e444-4995-b0a3-7a59477999d87163029522390179009.tmp"
[email protected] $ rm "/home/unetuser/~/cellnet/unet-12ffe596-e444-4995-b0a3-7a59477999d87163029522390179009.tmp"
[email protected]$ caffe_unet
Setting caffe_unet binary path to caffe_unet
Searching for caffe
[email protected]$ caffe
$ sftp "/home/sgrolab/Downloads/caffemodels/3d_cell_net_microspores-fluorescence_v0.modeldef.h5" "[email protected]:22:/home/unetuser/~/cellnet/unet-12ffe596-e444-4995-b0a3-7a59477999d8.modeldef.h5"
[email protected]$ caffe_unet check_model_and_weights_h5 -model "~/cellnet/unet-12ffe596-e444-4995-b0a3-7a59477999d8.modeldef.h5" -weights "~/data1" -n_channels 1 -gpu 0
F0310 19:40:44.387264    53 common.cpp:152] Check failed: error == cudaSuccess (35 vs. 0)  CUDA driver version is insufficient for CUDA runtime version
$ sftp "/home/sgrolab/Downloads/caffemodels/2d_cell_net_v0.caffemodel.h5" "[email protected]:22:/home/unetuser/~/data1"
[email protected]$ caffe_unet check_model_and_weights_h5 -model "~/cellnet/unet-12ffe596-e444-4995-b0a3-7a59477999d8.modeldef.h5" -weights "~/data1" -n_channels 1 -gpu 0
Model check failed:
F0310 19:40:56.398861    55 common.cpp:152] Check failed: error == cudaSuccess (35 vs. 0)  CUDA driver version is insufficient for CUDA runtime version
F0310 19:40:56.398861    55 common.cpp:152] Check failed: error == cudaSuccess (35 vs. 0)  CUDA driver version is insufficient for CUDA runtime version
Establishing SSH connection for '[email protected]:22'
Could not remove temporary file ~/cellnet/unet-12ffe596-e444-4995-b0a3-7a59477999d8.modeldef.h5: com.jcraft.jsch.JSchException: java.net.NoRouteToHostException: No route to host (Host unreachable)
Establishing SSH connection for '[email protected]:22'
Could not remove temporary folder /home/unetuser/~/cellnet: com.jcraft.jsch.JSchException: java.net.NoRouteToHostException: No route to host (Host unreachable)
Establishing SSH connection for '[email protected]:22'
Could not remove temporary folder /home/unetuser/~: com.jcraft.jsch.JSchException: java.net.NoRouteToHostException: No route to host (Host unreachable)
Establishing SSH connection for '[email protected]:22'
Establishing SSH connection for '[email protected]:22'
U-Net job aborted