Anyone applying the surface area metric as a diagnostic?

[cmhalffman]

MixSIAR conveniently incorporates the calculation of the Brett 2014 surface area metric as a diagnostic. I am trying to use the computed surface areas of my resource polygons (these include 4, 5, and 6-source scenarios) as a tool for evaluating model resolution. I can compare my MixSIAR calculated surface area to Figure 3 in Brett 2014, which shows "bias in outputs" vs. surface area for 3, 4, 5, and 6-source polygons; however, Brett's figure is based on regular, idealized, equal-sided polygons. Like many resource polygons, mine are NOT ideal; in fact, the 5- and 6-source polygons are concave (1 or more resources lie within the convex hull). I haven't seen any application of the surface area metric in the literature (beyond Brett's meta-analysis). Is anybody out there using the surface area metric, perhaps for less than ideal resource polygons, and, if so, what is your approach for evaluation?

[brianstock]

Hello,

Just wanted to share a couple thoughts on the surface area metric:

1. Note that the Brett (2014) metric is normalized surface area (area divided by source variance), and the calc_area function in MixSIAR differs from Brett (2014) by summing the source and TDF variances (since these are indistinguishable in the model). See details section of ?calc_area.

2. I don't think it's valid to calculate a normalized surface area, compare to Fig 3 in Brett (2014), and say that there will be X% bias. Brett used the code from FASTAR, which runs separate mixing models for each consumer data point individually, and then pools the p estimates. This greatly increases the weight of the uninformative/generalist prior (since the likelihood only has one data point), which Brett interprets as "bias". See also https://peerj.com/articles/5096/#p-66.

3. Even though I don't think Fig. 3 in Brett (2014) is useful, I do think calculating the surface area metric could be, which is why we added the calc_area function to MixSIAR.

4. One thing you could do with your data is a quick simulation to directly quantify the effect of the uninformative prior (assuming that's what you're using - informative prior would be better if you have auxillary info) on your proportion estimates. Could vary 1) the amount of mix/consumer data points included (subsample, duplicate), and 2) the source and TDF variance (so changing area metric), and see how the mean and SD of the proportion estimates respond.

[cmhalffman]

Thanks for the clarification, especially on the difference between the Brett (2014) normalized surface area vs. the MixSIAR surface area. Thanks too for your suggestions on quantifying the effect of the uninformative prior!