CycleGAN icon indicating copy to clipboard operation
CycleGAN copied to clipboard

__global__ function call is not configured

Open chesterliliang opened this issue 2 years ago • 1 comments

I recently tried to run he horse2zebra training as instructions in readme by 'DATA_ROOT=./datasets/horse2zebra name=horse2zebra_model th train.lua' --master brunch-- but got an tricky error as following:

---------- # Learnable Parameters --------------
G_A = 7845123
D_A = 2766529
G_B = 7845123
D_B = 2766529
------------------------------------------------
display_opt     {
  1 : "errG_A"
  2 : "errD_A"
  3 : "errRec_A"
  4 : "errI_A"
  5 : "errG_B"
  6 : "errD_B"
  7 : "errRec_B"
  8 : "errI_B"
}
#training iterations: 200
THCudaCheck FAIL file=/home/chester/torch/extra/cutorch/lib/THC/generic/THCTensorMathPairwise.cu line=59 error=52 : __global__ function call is not configured
/home/chester/torch/install/bin/luajit: ./models/cycle_gan_model.lua:186: cuda runtime error (52) : __global__ function call is not configured at /home/chester/torch/extra/cutorch/lib/THC/generic/THCTensorMathPairwise.cu:59
stack traceback:
        [C]: in function 'mul'
        ./models/cycle_gan_model.lua:186: in function 'fGx_basic'
        ./models/cycle_gan_model.lua:217: in function 'opfunc'
        /home/chester/torch/install/share/lua/5.1/optim/adam.lua:37: in function 'adam'
        ./models/cycle_gan_model.lua:236: in function 'OptimizeParameters'
        train.lua:137: in main chunk
        [C]: in function 'dofile'
        ...ster/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
        [C]: at 0x55a89ee24030

the code in THCTensorMathPairwise.cu is

THC_API void
THCTensor_(mul)(THCState *state, THCTensor *self_, THCTensor *src_, real value)
{
  THCAssertSameGPU(THCTensor_(checkGPU)(state, 2, self_, src_));
  if (self_ == src_) {
    if (!THC_pointwiseApply1(state, self_, TensorMulConstantOp<real>(value))) {
      THArgCheck(false, 2, CUTORCH_DIM_WARNING);
    }
  } else {
    THCTensor_(resizeAs)(state, self_, src_);

    if (!THC_pointwiseApply2(state, self_, src_, TensorMulConstantOp<real>(value))) {
      THArgCheck(false, 2, CUTORCH_DIM_WARNING);
    }
  }

  THCudaCheck(cudaGetLastError()); // here is line 59
}

The pretraind sample works well and I could run samples in cuda samples correctly. Here is my env

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Aug_15_21:14:11_PDT_2021
Cuda compilation tools, release 11.4, V11.4.120
Build cuda_11.4.r11.4/compiler.30300941_0

+-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... On | 00000000:01:00.0 On | N/A | | N/A 47C P8 19W / N/A | 46MiB / 16125MiB | 13% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 1070 G /usr/lib/xorg/Xorg 45MiB | +-----------------------------------------------------------------------------+

cuda runtime error 52  has less hint so I reallly hope someone could help.

chesterliliang avatar Oct 25 '21 07:10 chesterliliang

Sorry that we haven't worked with Lua/Torch for years and cannot help you debug it. Please use the PyTorch version, which is better supported.

junyanz avatar Mar 01 '22 21:03 junyanz