wekaDeeplearning4j icon indicating copy to clipboard operation
wekaDeeplearning4j copied to clipboard

Predictions are inverted (sometimes)

Open henrygouk opened this issue 6 years ago • 1 comments

Describe the bug I've noticed that the validation accuracy logged by dl4j is almost an exact inversion of the test set accuray reported by weka for some binary classification datasets I've been using.

To Reproduce Steps to reproduce the behavior:

  1. Download ARFF file from https://www.openml.org/d/1590
  2. Create a network in the weka explorer with three Dense layers of 100 units each, batch size of 100, everything else left as default default.
  3. See the accuracy reported by dl4j during training
  4. See the mismatch in test accuracy reported by weka

Note that this does not happen with all binary classification datasets I have used.

Expected behavior Accuracy reported by dl4j should be similar to what is reported by weka.

  • Weka version: 3-9-2-SNAPSHOT
  • wekaDeeplearning4j package version: v1.5.11
  • Operating System: Ubuntu 16.04

henrygouk avatar Jan 15 '19 05:01 henrygouk

I've come up with a temporary workaround: swapping the order that the two classes are defined in the ARFF file.

henrygouk avatar Jan 15 '19 06:01 henrygouk