Allow to change data passed via stanvars in brms backend per dataset

[martinmodrak]

Likely add an attribute to the data.frame in generated to pass this extra data.

Can be currently worked around by doing the conversion from brms to Stan model/data yourself and using a pure Stan backend.

[lunafazio]

Generated datasets and backend objects don't interact until compute_SBC, at which point the backend has already been compiled right? For my own workflow, I've built a wrapper in the past which accepted a function in the stanvars argument and bundled backend and compute in the same step while iterating over each generated dataset. But my impression is that properly integrating this into the package would require shifting a bunch of things around.

Maybe scan for the proposed attribute in template_data instead and let the user handle the mapping from generated datasets to a properly formatted stanvars argument?

[martinmodrak]

Not 100% sure what your aim is - do you need to do SBC for such model (and thus a workaround would be enough) or do you want to implement this to contribute for others? If you just need to run the SBC and if the actual Stan code is fixed, and the only changes via stanvars is to the data that's passsed to this Stan code, then this is already possible, although somewhat more annoying (you use brms::make_standata and brms::make_stancode to convert the brms model into Stan program and datasets and then use a normal Stan backend).

If the actual Stan code changes between than this is not supported and I don't think it would be sensible to support this - you can either run separate SBC for each version of the Stan code or modify the Stan code so that all the decisions are done at runtime based on data.

If you want to implement this, than my hunch is that a sensible design would be:

• In the generator, you add a stanvars_data attribute to template_data and to each generated dataset (which needs to be a data.frame). This implies you need a custom generator, which I think is OK, I think supporting this via SBC_generator_brms would need to be quite complex and does not make much sense anyway - if the extra data are not understood by brms how would you use brms to generate it?
• The stanvars_data attribute is checked so that a) it only contains data attributes and b) the generated Stan code when adding this stanvar is the same as the Stan code generated using the template_data (most likely this check will be in the brms backend)
• Those new stanvars are then added to all the other stanvars when calling make_standata.

Does that make sense?

[lunafazio]

I had to do this a while ago, but coming across this issue got me thinking about how it would be implemented so I don't have to use workarounds in case the need arises again. You're definitely right, there's more sensible ways of achieving what I had in mind, thank you for the insights!