AUROC reach peak in several epochs and then drop

[ZwkAlex]

auroc seems reach the peak in several epoch (5-10 epoch), then drop and never ascent after the peak, any reson for that? thanks so much.

[gathierry]

I have similar observations as well and I don't have a clear answer about that. Therefore, I put both results from best epoch and last epoch. I'll leave this issue open to see if there are better answers. And I might change the title to be more explicit.

[founderlin]

I had the same issue. Did anyone compare this version with the one in anomalib? The fastflow model in anomalib has a more stable and higher AUROC. Is there any tricks in anomalib?

I have similar observations as well and I don't have a clear answer about that. Therefore, I put both results from best epoch and last epoch. I'll leave this issue open to see if there are better answers. And I might change the title to be more explicit.

[cytotoxicity8]

It might be because of early stopping. Model architectures are almost identical.

[WiillyWonka]

I faced the same issue. I think the problem is in the log likelihood calculation for anomaly map construction. In this line https://github.com/gathierry/FastFlow/blob/2cf1f2f4c562a7f13cfb1959e3afe5df2f2d2565/fastflow.py#L148 the log likelihood must be calculated as -torch.mean(output**2, dim=1, keepdim=True) * 0.5 + log_jac_dets. This is consistent with the formula for variable substitution in the distribution. Also, it helped me. Now the learning process has become stable and doesn't drop.