AMET icon indicating copy to clipboard operation
AMET copied to clipboard

Apparent discrepency between code and documentation for fractional bias and fractional error

Open pavlovc2 opened this issue 2 years ago • 2 comments

The documentation states that the numerator and denominator should be summed separately to calculate fractional bias and error. image

However, the code for DomainStats in AQ_Misc_Functions.R appears to calculate these metrics as the sum after the fraction is calculated for each observation/model pair. The code also multiplies the result by a 100 to report as a percentage rather than a fraction.

https://github.com/USEPA/AMET/blob/1.5/R_analysis_code/AQ_Misc_Functions.R#L652

fb <- signif(c(fb,(sum((data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val)/2)))/length(data.df$stat_id))*100,3)
fe <- signif(c(fe,(sum(abs(data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val/2)))/length(data.df$stat_id))*100,3)

Am I correct that this is a discrepancy, and can this be updated to be consistent with the correct approach?

pavlovc2 avatar Jan 04 '24 16:01 pavlovc2

Nathan,

Thanks for your comment. It turns out the definition in the document is the one that is incorrect. The summing should be taking place outside of the brackets (as is done in the actual AMET R code). So, we will note this as a correction needed to this document.

Let me know if you have any further questions. Thanks!

Wyat

wkappel avatar Jan 04 '24 19:01 wkappel

Got it, thank you for the quick clarification!

pavlovc2 avatar Jan 04 '24 19:01 pavlovc2