What should two-argument categorical-family distributions do when the selected input changes?

[axch]

Consider the expression (categorical (simplex 0.5 0.5) (array (flip) (flip))). Suppose one coin is currently True and the other False, and the True one is currently selected. Suppose single-site m-h tries to propose to the True coin. The categorical choice will claim it can absorb. Suppose m-h proposes to change the coin value to False. Now there is no way to produce True by choosing one of False or False, but categorical already committed to absorbing, so it has no choice but to return -inf and force the transition to be rejected.

• [ ] What should be done here? Do we just eat the mixing rate slowdown?
• [ ] Dirichlet categorical variants crash in this situation; we should at least make them reject the transition. [Edit: this is now #451.]
• [ ] A one-option categorical will act as a freeze here, preventing all changes to the input. Do we want to a) disable giving categorical only one choice? b) teach one-choice categorical that it can't absorb changes? c) something else? In circumstances where the number of options is stochastic, would the value-dependent absorbing implied by (b) break detailed balance?

There are user-side workarounds:

• Only pass constant arguments to categorical and use them as lookup keys.
• Wrap categorical arguments in ref so that inference on their values does not constitute changing the arguments to the categorical.

[lenaqr]

Option: have categorical propagate instead of absorbing changes to the list of input choices? Then it's the same (ish) as what happens with if-expressions: usually the output will be resampled if the condition changes, unless the output is constrained. And if the categorical itself is constrained, then mixing among the choices is kind of sad but the workarounds are available.

Another option would be to make two-argument categorical equivalent to (lambda (simplex choices) (lookup choices (categorical simplex))), except for the part where lookup isn't invertible so this would force the use of refs.

[axch]

Can we do the same thing for the conjugate versions, instead of #453? Do we have a notion of "AAA on this argument but don't absorb at all on that one"? Alternately, do we also want to have the object that #453 would be the correct implementation of, perhaps under a different name?

[axch]

Particularly with non-conjugate categoricals, it feels sort of sad to give up on absorbing changes to the options, because as long as the selected option is there, the others ought to be free to shift around at will.