justcause icon indicating copy to clipboard operation
justcause copied to clipboard

Implement R-Pol policy risk score and ROC AUC score

Open MaximilianFranz opened this issue 5 years ago • 0 comments

Some datasets cannot be evaluated using the currently used scores PEHE or ENoRMSE, because

  1. No ground truth is available (e.g. the Jobs dataset from Lalonde)
  2. The classes are imbalanced and binary (e.g. the Twins dataset)

Thus, we need more scores for comprehensive evaluation. Especially the policy risk used, for example, by Shalit et al. Also, the ROC-curve or the area-under-the-curve (AUC) of the ROC-Curve should be used in binary cases like the wins dataset.

MaximilianFranz avatar Dec 01 '19 14:12 MaximilianFranz