Fitting, moments, and more

[sethaxen]

Has there been discussion on if/how fitting of measures from draws should be implemented? I'm also wondering if/how estimators of moments from draws or exact moments when known should be included. In most cases on manifolds, these are not known, but for cases where they are, it would be nice to include them.

[cscherrer]

Not really. There's a Likelihood that I think can be really nice. We could do something like Distributions (even calling that code when it helps), but we can also take a pointwise product (⊙) of a measure with a likelihood. In conjugate cases this pointwise product can reduce so there's a closed-form solution. I imagine it could even be possible to have generated functions solve for zero gradient symbolically for a given structure, so you pay once and then future calls are very fast. But that's longer term :)

[gdalle]

Just bumping this to say that I would really love a fit interface in MeasureTheory that plays nice with Distributions, but also enables us to do more general things like MAP instead of MLE. For now, my PointProcesses code is full of fitting functions that must take a prior as additional argument, like fit(::Categorical, prior::Dirichlet, x), but I'm wondering if we could be more clever here.

[gdalle]

I think it is feasible to add a few lines to distproxies.jl and extend the fit function in a quick and dirty way. The only challenge is that we need a way to associate each MeasureTheory.jl type to its Distributions.jl type. The least worst way I can think of is to maintain a list of all the measure symbols inspired by Distributions.jl, and to do something like

for f in [:fit, :fit_mle, :suffstats]
    for D in [:Normal, :Exponential, ...]
        @eval begin
            import Distributions: $f
            export $f
            Distributions.$f(::Type{$D}, args...) = Distributions.$f(::Type{Distributions.$D}, args...)
        end
    end
end

@cscherrer any advice? Ihis is the first time I'm venturing into metaprogramming so my self-confidence is low ^^

[cscherrer]

Great idea @gdalle ! It seems like we could get this to work, but we'll need some extra steps. I think I'd first get suffstats to work. If we call fit_mle, we'll end up with a Distribution, but we want a measure. And this measure might have a parameterization not available in Distributions. So I think we'll need to adapt fit_mle accordingly.

[gdalle]

I'll get to work on that. Btw, do we really need both the DistributionMeasure type and our own rip-offs from Distributions.jl?