Rule add-ons (such as scale factors)

[bartvanerp]

With our recent acceptance to the SiPS conference on Efficient Model Evidence Computation in Tree-structured Factor Graphs, we would like to formalize the use of scale factors as prototyped in the dev_scalefactors branch. These scale factors basically specify the (log-)scaling of messages, which can be used for tracking the model evidence. Current implementations are limited to exact message passing procedures (sum product/belief propagation).

In the dev_scalefactors branch, scale factors have been implemented by creating messages ScaledMessage, which contain both the distribution as the scaling. The major downside of this approach is that it requires duplicating rules for this particular type of message. This adds a lot of clutter to the update rules and should be simplified.

After a discussion we propose the following rule specification:

@rule Node(messages, meta, options) = begin
    # compute parameters
    params = f(messages)
    @scaling begin
        # compute scale
        scale = g(messages, params)
        return scale
    end
    return new_message(params)
end

Inside the rule we can introduce a @scaling block, which specifies how the scaling gets updated in a factor node. This @scaling block will not be present in all rules, but just in a select few. Importantly, for efficiency purposes, this scaling block can use intermediate quantities of the message computation. The options argument can be used as a flag for the @rule macro to decide whether it wants to use the scaling update. It should only be used on command.

As discussed, it would be nice if this addition could be generalized not only to @scaling, but also to @somethingelse. This features should be enabled globally in the @model macro as [ extras = Scaling() + Something() ]. Whether we term this extras, add-ons or something else is up for debate.

This feature will also extend the Message (and Marginal) structs to something like

struct Message{D, E}
    distribution :: D
    extra :: E
end

where extras is nothing by default.

Preferable, the inference algorithm throws an error if a @scaling block is missing. Nodes (such as priors), with clamped interfaces, will always have a default scaling of 1 (or log-scaling of 0).