Finding a sub-optimal threshold without using testing data

[eladmeir]

As the paper states that Tau, is predefined, I have come to a "real world application problem" - setting the threshold under the absence of any abnormal data, like we'd expect from a one class classifier to perform in real world (otherwise - why not using the abnormal data for training?)

Has anyone succeed in finding a suitable thresholding extraction method for this procedure, or am I missing something here?

P.S. The implementation has a nice-to-simulate threshold extraction procedure which uses the "correct" labels provided at testing time in order to find some meaningful point on the ROC graph, which is not possible in real world.

It is trivial to see that this method is highly dependent on the "correct" labels, given at testing time, by a shuffle of instances between the 2 classes (normal <-> abnormal), which easily changes the predictions of all "1"s

But obviously this cannot be done while predict an image of an object of an unknown class...