MixSIAR icon indicating copy to clipboard operation
MixSIAR copied to clipboard

Calculating the Bayesian R-squared

Open naalipalo opened this issue 3 years ago • 0 comments

I submitted my Isotope paper for review and one comment was regarding my MixSIAR models and how well the top model performed. They suggested I calculate an R2 value like one might if you used AIC to pick the best model and then describe how that current model fits that particular data. One issue I have with my data is that my "best" model isn't much better than any of the other models above the Null. the difference in the SE of LOOic overlap `

Model Covariate(s) LOOica se_LOOicb dLOOicc se_dLOOicd
13 Harvest method by Age 282.2 29 0 NA
5 Harvest method 284.9 30.4 2.7 7.5
3 Location 285.6 28 3.4 8.3
7 Harvest method by Sex 285.9 30.5 3.7 8.3
6 Harvest method by Location 286.7 29.9 4.5 7
2 Sex 288.1 28.8 5.9 10.1
1 Null 288.7 28.5 6.5 9.2
` What I have said was: `When assessing the diet of spring harvest, the models suggests that harvest method by age class had some explanatory power regarding the food consumption patterns (Table 3). However, there is uncertainty in the reliability of the “best” model because of overlapping SE of the deviation (se_dLOOic; Table 3) of models ranked above the null. As such, sex and location may hold some additional explanatory power, though not enough to indicate strong relationships.`

Though Im not sure if I am phrasing that correctly. The difference in SE is like confidence intervals and therefore because those overlap, we can't distinguish really the "best". Anyway, The rest of my paper is predicated upon the fact that although the others have some explanatory value, harvest method is clearly important and we proceed with just exploring the patters observed from age class. So, because of that, the reviewer wants to see how well that one model is doing and is asking for an R-squared value type indication.

Has anyone done this yet? I found https://avehtari.github.io/bayes_R2/bayes_R2.html and the associated paper https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1549100 but do not know how to write the code to incorporate this into the example coding from MixSIAR. unless Ive missed it somewhere. Otherwise, I don't know how to tell this reviewer R2 isn't exactly transferrable to Bayesian processes. R2 evaluates a particular point and doesn't deal with uncertainty

naalipalo avatar Oct 08 '21 01:10 naalipalo