New idea.

[YagaoDirac]

Update: I tried some Gaussian like function as activations, some resulted well. If the key is the bell-shape, then I guess anything that gives out the bell shape can work also well enough.

Original comment: If you see this, comment anything to let me know. Thanks. I saw the activation, it's interesting. I tried this x = np.linspace(-5, 5, 201) sig = 3#2#1#0.1#0.3#0.5 l = 6#4#2

numerator = -np.power(x, ll) denominator = 2sig*sig result = np.exp(numerator/denominator)

fig, ax = plt.subplots(1) ax.plot(x, result)

If this is exactly what in your code, then I guess the "result*cos(x)" can result better. Good luck.