Rational quadratic spline identity init

[Olllom]

First of all thanks for providing this awesome library.

In our applications, it is sometimes beneficial to initialize transforms as the identity. We usually achieve this by setting all parameters to zero (e.g., shift and logscale in an affine coupling layer).

In your implementation of the rational quadratic spline, we can achieve this by replacing the line

https://github.com/bayesiains/nflows/blob/639c3a771d57c29a27c307140cc94a1008ee9f55/nflows/transforms/splines/rational_quadratic.py#L97

by something like

    import numpy as np
    ...
    mean_slope = (top-bottom) / (right-left)
    derivatives = mean_slope * F.softplus(unnormalized_derivatives, beta=np.log(2))
    derivatives =  derivatives.clip(min_derivative, None)

Is this something you would consider changing in your implementation? Otherwise we can just go ahead and use an adaptation of your rq-spline code in our repo (of course with the appropriate references to your implementation).

Cheers, Andreas

Tagging @invemichele @jonkhler

[arturbekasov]

Hi Andreas,

This sounds sensible. I don't have the time these days to implement something like this myself, unfortunately, but would be willing to review a PR. Maybe

derivatives = min_derivative + F.softplus(unnormalized_derivatives, beta=(np.log(2) / (1 - min_derivative)))

would be even better as it avoids clipping. To keep the transform backwards compatible I'd make beta a parameter to the constructor, keeping at 1 by default, although there might be a more sensible way of parametrizing this behaviour.

Cheers,

Artur

[arturbekasov]

#65 to resolve this has just been merged.