NNlib.jl

NNlib.jl copied to clipboard

Currently NNlib exports σ, which in this package refers to the sigmoid function. The name should be changed to "sigmoid" to avoid accidental usage in other code. For example the following code will silently use sigmoid (in this example instead of softmax) without raising an error:

julia> using Flux
struct MyOp{F, M<:AbstractMatrix}
  v::M
  σ::F
end
function (a::MyOp)(x::AbstractVecOrMat)
  return σ(a.v * x)
end
a = MyOp(rand32(3,3), Flux.softmax);
x = rand32(3,3);
a(x)
julia> a(x) .- a.σ(a.v*x)
3×3 Matrix{Float32}:
 0.513941  0.363848  0.456001
 0.343419  0.281814  0.289344
 0.519608  0.371748  0.457464