stan-dev
document sum to zero matrix
Once https://github.com/stan-dev/math/pull/3169 is in I'll need to document this.
Doc thoughts
The sum_to_zero_matrix is a two-dimensional generalization of the sum_to_zero_vector, using the same Helmert-type transform to enforce the columns to be mutually orthonormal in $\mathbb{R}^{ (N + 1) \times (M + 1) }$.
- Define a 1D Helmert basis in dimension (N+1) by
$$
\boxed{
u_{k}(i)
\ =
\begin{cases}
1/\sqrt{k \ (\ k+1 )}, & \text{if } i \le k,\
-k/ \sqrt{k\ (\ k+1 \ )}, & \text{if } i = k+1,\
0, & \text{if } i > k+1,
\end{cases}
\quad
\text{for }k=1,\ldots, N, \ \ i=1,\ldots, N+1.
}
$$
- Define a 1D Helmert basis in dimension (M+1) by an exactly analogous formula:
$$
v_{\ell}(j)
\ =
\begin{cases}
1/\sqrt{\ell,(,\ell+1,)}, & \text{if } j \le \ell, \
-\ \ell/ \sqrt{\ell,(\ \ ell+1 )}, & \text{if } j = \ell+1, \
0, & \text{if } j > \ell+1,
\end{cases}
\quad
\ell=1,\ldots,M,\ \ j=1,\ldots,M+1.
$$
- Form the 2D basis by tensor‐products:
$$
h_{k,\ell}(i,j)
\ =
u_{k}(i)
\ \times
v_{\ell}(j),
$$
giving $k=1, \ldots,N$ and $\ell=1,\ldots,M$. Each $h_{k,\ell}$ is an $(N+1) \ \times \ (M+1)$ “contrast pattern” with row and column sums $= 0$. These $NM$ matrices ${h_{k,\ell}}$ are orthonormal in $\mathbb{R}^{(N+1)\times(M+1)}$.
If one wanted a ND basis then forming the ND tensor-product of the Helmert bases will result in the correct orthonormal transform.
Below is one possible way to present the mathematics behind your Helmert‐like row‐and‐column‐constraining transform in a style reminiscent of a Stan User Guide, but without actual C++ code. The loops are written purely in mathematical notation, showing how each element of the output (Z) is computed.
Mathematical Description of the Transform
We have an input matrix
$$ x \in \mathbb{R}^{N \times M}, $$
and we wish to construct an output matrix
$$ Z \in \mathbb{R}^{(N+1)\times(M+1)}, $$
subject to the row‐sum‐zero and column‐sum‐zero constraints. Define vectors
$$ \beta \in \mathbb{R}^N, \quad \beta_i = 0 \quad \text{for } i = 0,\dots, N-1, $$
and initialize
$$ Z_{p,q} = 0 \quad \text{for all } 0 \le p \le N, \quad 0 \le q \le M. $$
Then proceed as follows:
Descending loop over columns $j = M-1, M-2, \ldots, 0$:
Define
$$ a_j = \frac{1}{\sqrt{(j+1)(j+2)}}, \quad b_j = (j + 1),a_j. $$
Initialize an accumulator $ \mathrm{ax_prev} = 0 $.
Set
$$ Z_{N,j} = 0. $$
Inside each column, descending loop over rows $i = N-1, N-2, \dots, 0$:
Define
$$ a_i = \frac{1}{\sqrt{(i+1)(i+2)}}, \quad b_i = (i + 1),a_i. $$
Form the partial combination
$$ b_i , x_{i,j} - \mathrm{ax_prev} = b_i , x_{i,j} - \mathrm{ax_prev}. $$
Denote this quantity by $b_{i_x}$.
Update the output:
$$ Z_{i,j} = b_j ,\bigl(b_{i_x} bigr) - \beta_i. $$
Update the $\beta$ vector:
$$ \beta_i \leftarrow \beta_i + a_j ,\bigl(b_{i_x} \bigr). $$
Impose the row‐sum and column‐sum constraints:
$$ Z_{N,j} \leftarrow Z_{N,j} - Z_{i,j}, \quad Z_{i,M} \leftarrow Z_{i,M} - Z_{i,j}. $$
Accumulate
$$ \mathrm{ax_prev} \leftarrow \mathrm{ax_prev} + a_i , x_{i,j}. $$
After finishing all rows $i$ for a given column $j$, update
$$ Z_{N,M} \leftarrow Z_{N,M} - Z_{N,j}. $$
This ensures the bottom‐right corner reflects the column sum constraint in the last row.
At the end of these nested loops, the matrix $Z$ satisfies
$$ \sum_{k=0}^N Z_{k,j} = 0 \quad \forall, j=0,\dots,M, \quad \text{and} \quad \sum_{\ell=0}^M Z_{i,\ell} = 0 \quad \forall, i=0,\dots,N. $$
That is, each row and column of $Z$ sums to zero. The particular constants ${a_i, b_i}$ and ${a_j, b_j}$ ensure an orthonormal (Helmert‐type) structure when viewed as a linear operator from $\mathbb{R}^{N\times M}$ to $\mathbb{R}^{(N+1)\times(M+1)}$.
As an additional note: besides just the new type, we will also want to document the new overloads to functions like sum_to_zero_constrain on their page
It's all merged now -- @spinkney do you want to take the first crack? I can add the new function overloads to whatever branch you work on
While we're at it, there's a typo in the sum to zero vector: https://discourse.mc-stan.org/t/constraint-transforms/39384
The sqrt of y fix is merged but the docs need to be rerendered to show it
On Tue, May 6, 2025, 11:18 AM Brian Ward @.***> wrote:
WardBrian left a comment (stan-dev/docs#867) https://github.com/stan-dev/docs/issues/867#issuecomment-2854975616
While we're at it, there's a typo in the sum to zero vector: https://discourse.mc-stan.org/t/constraint-transforms/39384
— Reply to this email directly, view it on GitHub https://github.com/stan-dev/docs/issues/867#issuecomment-2854975616, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFU3D6NONLYMUX6WDUJTXHT25DHELAVCNFSM6AAAAABZYNWNU6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNJUHE3TKNRRGY . You are receiving this because you were mentioned.Message ID: @.***>