Power analysis using UC or pyBats dynamic linear models

[MilaimKas]

Hello there, first, many thanks for providing a stable and Bayesian based python version of the original R package.

I was previously using the deprecated pycausalimpact package to perform analysis of intervention. Sure it was not Bayesian but was nevertheless providing some useful insights. In addition, I used to perform "power analysis" (so far as this term can be used in the context of time series) to assess the "power" of my model, i.e. to answer the question "what effect size would I be able to detect at a given level of confidence". In order to do that, I would:

• define a pre- and post-period
• define a list of effect sizes
• loop over the effect size ** add the effect to the post-period (optionally including a lag and noises) ** extract the final result (last point of the estimated cumulative effect size and CI)
• plot the real effect against the measured effect with the CI

I would then obtain a plot similar to this: Where the blue filled region correspond to the 95% interval and the gray to the 80%. I would then interpret the results as follows: at a 95% level of confidence I would be able to "detect" an effect of ~6% or larger, at a 80% level of confidence I would be able to detect a very small effect (~1%).

Of course, this is a typical frequentest approach, but I still found it very useful to assess the "power" of a model. In addition, since I often encountered the case where it was possible to "tune" the model (varying the pre and post period length, adding or removing certain control variables) to get significant or non-significant result, I also used the same approach described above but, in addition to looping over effect size, I would also loop over pre- and post-period lengths. This would give me an idea on how stable the results are with respect to these parameters.

This approach needs many of causal impact calculations, which is not possible with the current tensor flow based model. I was therefore wondering how difficult it would be to allow the user to switch from a tensor flow bsts model to, for example, a UC model ? The idea would be to perform these "power analysis" with the simple model, decide on which parameters to use, and asses the final impact using the more accurate tensor flow model. Alternatively, to stay within the Bayesian framework, one could also rely on Normal approximations and used conjugated priors to get the posteriors. I believe, this is what the python package pyBats does.