NeuralNetwork.NET icon indicating copy to clipboard operation
NeuralNetwork.NET copied to clipboard

Added Mish activation function - Ready for merge

Open JamesLear92 opened this issue 4 years ago • 0 comments

I've added the Mish activatation function as suggested in #93

~~This is work in progress at the moment as it's working perfectly when not using CUDA.~~

~~When running the DigitsCudaTest sample with the Mish function, I'm currently seeing:~~

 System.Exception: 'methodBody is null
 Source location stack:
-> at NeuralNetworkNET.Networks.Activations.ActivationFunctions.[Single MishPrime(Single)]
-> at NeuralNetworkNET.cuDNN.CuDnnExtensions+<>c__DisplayClass1_0.[Void <ActivationBackward>g__Kernel|0(Int32)]
-> at Alea.Parallel.Device.DeviceFor.[Void Kernel(Int32, Int32, System.Action`1[System.Int32])]
-> at defining runtime64 (sm52,64bit)

~~MishPrime function looks right to me (and works fine without cuda), but evidently something is wrong. I wonder if this is something Alea need to support.~~

EDIT: Now working, and ready for merge. Issue was caused by MishPrime function using Math.Pow (unsupported).

JamesLear92 avatar Nov 22 '19 20:11 JamesLear92