Figure out adjacency append error

[ejolly]

This hasn't resulted in any test errors but I think I may have formatted something weird here, because vscode gives me an error that dm isn't defined in this else statement: https://github.com/cosanlab/nltools/blob/master/nltools/data/design_matrix.py#L448

Just noting down to figure out for future (and probably add more design matrix tests!)