where is the Relu function in the training code?

[gx2017]

Hi Dr.Xu,

In the step1_DNNenh_for 16kHz.m file, i found you use relu function to do decoding, however, i don't found relu function in the training code, if i want to use relu function as active function, should i add it myself?

[yongxuUSTC]

ReLU can be added into DevFunc.cu:

////////ReLU global void kernSigmoid(int n, float* in_vec, float* out_vec) { int i = (blockIdx.x * blockDim.x) + threadIdx.x; if (i < n) //sigmoid //out_vec[i] = 1.0f/(1.0f + expf(- in_vec[i]));

  //ReLU
  if(in_vec[i]>0)
		out_vec[i]=in_vec[i];
	  else
		out_vec[i]=0.0f;

}

global void kernDsigmoid(int n, float* in_vec, float* out_vec) { int i = (blockIdx.x * blockDim.x) + threadIdx.x;

if (i<n)
{
	//sigmoid
		//const float y = in_vec[i];
		//out_vec[i] = (1.0f - y) * y;
  
  //ReLU
  if(in_vec[i]>0)
		out_vec[i]=1.0f;
	  else
		out_vec[i]=0.0f;
}

}

[yongxuUSTC]

The ReLU has already been updated in the code now.