Adding Sparse Matrix Indexing

[SteveBronder]

Summary:

Looking over the Eigen docs I think we can support the following in the stan language

data {
  int N; // rows
  int M; // cols
  int nz_vals; // number of nonzero values
  int nz_vec_vals; // number of nonzero values in vector
  int nz_rows[nnz_vals]; // row position of nonzero values
  int nz_cols[nnz_vals]; // col position of nonzero values
  int nz_vec_rows[nnz_vals]; // row position of nonzero vector values
  matrix[N, M] A_mat; // basic matrix
  sparse_matrix[nz_rows, nz_cols, N, M] A_sparse; // sparse matrix
  sparse_vector[nz_vec_rows, N] A_sparse_vec; // sparse vector
}

transformed data {
  sparse_matrix[N, M] A_sparse_mod = A_sparse;
  sparse_vector[N] A_sparse_vec_mod = A_sparse_vec;

  // Coeff access anywhere (slow but doable, fine for preallocated nnz areas)
  A_sparse_mod[1, 5] = 10.0;
  // Row / Column assignment:
  //     From sparse vec 
  A_sparse_mod[1, ] = sparse_vector; // stored as column major so efficient
  A_sparse[, 1] = sparse_vector; // bad performance but low effort
  //    From normal mat (possible but why?)
  A_sparse_mod[1:10, 1:10] = A_mat[1:10, 1:10]; 
  A_sparse_mod[1, ] = A_mat[1, ];
  A_sparse_mod[, 1] = A_mat[, 1];
  //  
}

@avehtari which of these would we want? All of them? Am I missing any?

Current Version:

v2.22.0

[avehtari]

I'm fine with these. We discussed also rbind and cbind type functions, but I guess it's not indexing. Pinging @dpsimpson, too.

[SteveBronder]

Cool cool! Yeah I would add rbind and cbind to stan math

[bob-carpenter]

Will a dense to sparse conversion lose derivative information on the zero entries?

We have append_row and append_col instead of rbind and cbind. What signatures are you considering adding?

[SteveBronder]

Will a dense to sparse conversion lose derivative information on the zero entries?

Do you mean like

  A_sparse_mod[1:10, 1:10] = A_mat[1:10, 1:10]; 

For that I would preserve the zero value entries so it would retain the derivative information

[bob-carpenter]

Yes. That was the question. If it's data, you can drop the zeros, but if it's parameter, you need to keep all those in the sparse form.

[SteveBronder]

Yes. That was the question. If it's data, you can drop the zeros, but if it's parameter, you need to keep all those in the sparse form.

Cool! Yeah dropping would be better for data but we can do that optimization later.

Also good news! Turns out the stuff I did in #2893 to generalize the rvalue and assign functions make them all pretty much just work on sparse matrices. I'm working on the tests now to verify

[dpsimpson]

I got confused for a moment by nnz_rows which I interpreted as "number of non-zero rows", not the rows that have non-zeros in them (it's the first n that's the problem). Maybe nonzero_rows or row_index would be clearer.

Still not a fan of dense -> sparse (or sparse -> dense) assignment.

[SteveBronder]

I got confused for a moment by nnz_rows which I interpreted as "number of non-zero rows", not the rows that have non-zeros in them (it's the first n that's the problem). Maybe nonzero_rows or row_index would be clearer.

Fixed!

Still not a fan of dense -> sparse (or sparse -> dense) assignment.

I'm fine with not allowing it, in stanc3 I can write the assignment so that it only works for sparse->sparse