Allow ForwardDiff?

[hanjo-kim]

I was tinkering around with this package and saw that the package doesn't allow Dual Numbers which leads to an error when trying to use ForwardDiff.

The following code

using Distributions, MvNormalCDF, Random
d = MvNormal([0;0], Σ); 
f(x) = mvnormcdf(d, [x; -Inf], [Inf; -0.2])[1]
ForwardDiff.derivative(f, 0.1)

leads to the following error:

ERROR: MethodError: no method matching _chlrdr!(::Matrix{Float64}, ::Vector{Forw
ardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1}}, ::Vector{Float64
})

Closest candidates are:
  _chlrdr!(::AbstractMatrix{T}, ::AbstractVector{T}, ::AbstractVector{T}) where 
T<:Real
   @ MvNormalCDF C:\Users\hjkim\.julia\packages\MvNormalCDF\Q42gl\src\functions.
jl:339

It looks like _chlrdr! function only accepts T<:Real. I think the fix would be relatively simple(?). Maybe get rid of T<:Real, and have the outputs be something like zeros(eltype(input)) or something, but I'm not an expert at Julia.

Is there any plans to make it usable with ForwardDiff any time soon? Thanks!

[PharmCat]

I'll try to do this.

[PharmCat]

Try to check dev branch

[hanjo-kim]

hmm I'm still getting an error...

My current version of MvNormalCDF is

 [37188c8d] MvNormalCDF v0.3.0 `https://github.com/PharmCat/MvNormalCDF.jl#dev`

so I'm using the dev branch.

The error message that I get is

ERROR: MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1})

Closest candidates are:
  (::Type{T})(::Real, ::RoundingMode) where T<:AbstractFloat
   @ Base rounding.jl:207
  (::Type{T})(::T) where T<:Number
   @ Core boot.jl:792
  (::Type{T})(::AbstractChar) where T<:Union{AbstractChar, Number}
   @ Base char.jl:50
  ...

Stacktrace:
  [1] convert(#unused#::Type{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1})
    @ Base .\number.jl:7
  [2] setindex!(A::Vector{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1}, i1::Int64)
    @ Base .\array.jl:969
  [3] _chlrdr!(Σ::Matrix{Float64}, a::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1}}, b::Vector{Float64})
    @ MvNormalCDF C:\Users\hjkim\.julia\packages\MvNormalCDF\hZbcL\src\functions.jl:426
  [4] qsimvnv!(Σ::Matrix{Float64}, a::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1}}, b::Vector{Float64}, m::Int64, rng::Random.RandomDevice)
    @ MvNormalCDF C:\Users\hjkim\.julia\packages\MvNormalCDF\hZbcL\src\functions.jl:198
  [5] mvnormcdf(μ::Vector{Float64}, Σ::PDMats.PDMat{Float64, Matrix{Float64}}, a::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1}}, b::Vector{Float64}; m::Int64, rng::Random.RandomDevice)
    @ MvNormalCDF C:\Users\hjkim\.julia\packages\MvNormalCDF\hZbcL\src\functions.jl:182
  [6] mvnormcdf
    @ C:\Users\hjkim\.julia\packages\MvNormalCDF\hZbcL\src\functions.jl:126 [inlined]
  [7] #mvnormcdf#1
    @ C:\Users\hjkim\.julia\packages\MvNormalCDF\hZbcL\src\functions.jl:71 [inlined]
  [8] mvnormcdf(dist::Distributions.FullNormal, a::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1}}, b::Vector{Float64})
    @ MvNormalCDF C:\Users\hjkim\.julia\packages\MvNormalCDF\hZbcL\src\functions.jl:70
  [9] f(x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Float64}, Float64, 1})
    @ Main .\REPL[5]:1
 [10] derivative(f::typeof(f), x::Float64)
    @ ForwardDiff C:\Users\hjkim\.julia\packages\ForwardDiff\vXysl\src\derivative.jl:14
 [11] top-level scope
    @ REPL[6]:1

[PharmCat]

hmm I'm still getting an error...

Thank you! Can you give minimal working example?

Because this seems works:

using MvNormalCDF
using ForwardDiff
 μ = [1., 2., 3.] 
  Σ = [1 0.25 0.2; 0.25 1 0.333333333; 0.2 0.333333333 1]
  ag = [-1; -4; -2]
  bg = [1; 4; 2]
  gf(x) = MvNormalCDF.mvnormcdf(x, Σ, ag, bg)[1]
  ForwardDiff.gradient(gf, [1, 2, 3])

May be last commit will work.

[hanjo-kim]

I think taking derivatives wrt the mean vector or elements works great, but I'm getting an error when taking derivatives wrt to covariance matrices or supports. I could be missing something... I tried updating the package before running this code.

Here are the examples: Taking derivatives wrt to the variance-covariance matrix (elements or the entire matrix) doesn't work;

using MvNormalCDF
using ForwardDiff
f(x) = MvNormalCDF.mvnormcdf([0.;0.], reshape(x, 2, 2), [-1.; -1.], [1.; 1.])[1]; 
ForwardDiff.gradient(f, [0.1; 0.; 0.; 0.1])

Element-wise:

using MvNormalCDF
using ForwardDiff
f(x) = MvNormalCDF.mvnormcdf([1.;2.], [x 0.; 0. 0.1], [-1.; -1.], [1.; 1.])[1]; 
ForwardDiff.derivative(f, 0.1)

Also, seem to be getting an error when taking derivative wrt bounds:

using MvNormalCDF
using ForwardDiff
f(x) = MvNormalCDF.mvnormcdf([0.;0.], reshape([0.1; 0.; 0.; 0.1], 2, 2), [x; -1.], [1.; 1.])[1]; 
ForwardDiff.derivative(f, -1.)

All the error messages seem to be MethodError.

ERROR: MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.T
ag{typeof(f), Float64}, Float64, 1})

Closest candidates are:
  (::Type{T})(::Real, ::RoundingMode) where T<:AbstractFloat
   @ Base rounding.jl:207
  (::Type{T})(::T) where T<:Number
   @ Core boot.jl:792
  (::Type{T})(::AbstractChar) where T<:Union{AbstractChar, Number}
   @ Base char.jl:50
  ...

Stacktrace:
 [1] convert(#unused#::Type{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeo
f(f), Float64}, Float64, 1})
   @ Base .\number.jl:7
 [2] setindex!(A::Vector{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(f
), Float64}, Float64, 1}, i1::Int64)
   @ Base .\array.jl:969
 [3] _chlrdr!(Σ::Matrix{Float64}, a::Vector{ForwardDiff.Dual{ForwardDiff.Tag{ty
peof(f), Float64}, Float64, 1}}, b::Vector{Float64})
   @ MvNormalCDF C:\Users\hjkim\.julia\packages\MvNormalCDF\hZbcL\src\functions
.jl:426

Thank you for your help!

[PharmCat]

I was tinkering around with this package and saw that the package doesn't allow Dual Numbers which leads to an error when trying to use ForwardDiff.

In think last commit on dev should work.

[hanjo-kim]

It works great. Thank you for your help.

[CarloLucibello]

can this be closed?