NeuralNetwork.NET
NeuralNetwork.NET copied to clipboard
Added Mish activation function - Ready for merge
I've added the Mish activatation function as suggested in #93
~~This is work in progress at the moment as it's working perfectly when not using CUDA.~~
~~When running the DigitsCudaTest sample with the Mish function, I'm currently seeing:~~
System.Exception: 'methodBody is null
Source location stack:
-> at NeuralNetworkNET.Networks.Activations.ActivationFunctions.[Single MishPrime(Single)]
-> at NeuralNetworkNET.cuDNN.CuDnnExtensions+<>c__DisplayClass1_0.[Void <ActivationBackward>g__Kernel|0(Int32)]
-> at Alea.Parallel.Device.DeviceFor.[Void Kernel(Int32, Int32, System.Action`1[System.Int32])]
-> at defining runtime64 (sm52,64bit)
~~MishPrime function looks right to me (and works fine without cuda), but evidently something is wrong. I wonder if this is something Alea need to support.~~
EDIT: Now working, and ready for merge. Issue was caused by MishPrime function using Math.Pow (unsupported).