Add Delta node with Extended and Unscented transform

[albertpod]

This PR implements two types of approximation of messages around the Delta node, i.e. DeltaExtended (ET) and DeltaUnscented (UT). Both ET and UT approximate the messages with a Gaussian. To activate the approximations, the user must provide the approximation method inside the meta, e.g.

z ~ f(x) where {meta = ET()}

Both approximations have been ported from ForneyLab.jl.

ET computes updates by local linearization of the Delta node-function. UT computes updates by deterministic sampling approach.

If inverse functions are not specified in meta, e.g. ,

y ~ f(x, z) where {meta = UT()}

as opposed to

y ~ f(x, z) where {meta = UT(inverse=(f_x, f_z)}

then backward messages are computed by a Rauch-Tung-Striebel smoother approach, in accordance with (Petersen et al., 2018, "On approximate Gaussian Message Passing on Factor Graphs").

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov-commenter]

Codecov Report

Base: 76.72% // Head: 77.50% // Increases project coverage by +0.78% :tada:

Coverage data is based on head (9f5e216) compared to base (aa8a2ad). Patch coverage: 84.60% of modified lines in pull request are covered.

Additional details and impacted files

@@                Coverage Diff                @@
##           develop-delta     #213      +/-   ##
=================================================
+ Coverage          76.72%   77.50%   +0.78%     
=================================================
  Files                203      213      +10     
  Lines               6723     7176     +453     
=================================================
+ Hits                5158     5562     +404     
- Misses              1565     1614      +49     

Impacted Files	Coverage Δ
src/ReactiveMP.jl	66.66% <ø> (ø)
src/helpers.jl	64.39% <0.00%> (-1.00%)	:arrow_down:
src/message.jl	70.88% <0.00%> (-0.55%)	:arrow_down:
src/nodes/flow/flow.jl	100.00% <ø> (ø)
src/nodes/gamma_mixture.jl	3.88% <0.00%> (-0.42%)	:arrow_down:
src/rules/flow/in.jl	100.00% <ø> (ø)
src/rules/flow/marginals.jl	100.00% <ø> (ø)
src/rules/flow/out.jl	100.00% <ø> (ø)
src/rules/gamma_mixture/switch.jl	0.00% <ø> (ø)
src/rules/normal_mixture/out.jl	100.00% <ø> (ø)
... and 24 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[bvdmitri]

@albertpod @abpolym master branch now has fixes for the failing documentation build on CI

[bvdmitri]

@albertpod @ThijsvdLaar @bartvanerp I've mode some major changes to this branch, you might take a look into it once again if you want. Most of my changes were aimed to make the code faster and allocate less. I also reused more code for approximations in different places.

There is still a room for improvement, but together with Albert we've decided to merge it first and make more improvements later. In my opinion, the PR is complete and ready for merge.

[bvdmitri]

@albertpod Lets not forget to change DeltaFn to Delta in the upcoming "big" delta branch

[bvdmitri]

The branch has been merged into the develop-3.0-delta, but we keep this PR open for now in case someone has more comments. We should keep it open before we release.

[bvdmitri]

Merged in https://github.com/biaslab/ReactiveMP.jl/commit/80e919af65b7bc4a4831b7756b0f7dd2b6f9cd62