[whales, dry_eval] "Sorry, but this score is not high enough to pass the task" in classification

[rafajak]

neptune run -- dry_eval --problem whales

The above throws an error after evaluating the classification algorithm [ie. doesn't pass the task], and then goes back to command prompt.

2018-04-03 10-22-58 minerva >>> running: classification
2018-04-03 10-23-00 minerva >>> step classifier_encoder adapting inputs
2018-04-03 10-23-00 minerva >>> step classifier_encoder loading...
2018-04-03 10-23-00 minerva >>> step classifier_encoder transforming...
2018-04-03 10-23-00 minerva >>> step classifier_loader adapting inputs
2018-04-03 10-23-00 minerva >>> step classifier_loader loading...
2018-04-03 10-23-00 minerva >>> step classifier_loader transforming...
2018-04-03 10-23-00 minerva >>> step classifier_network unpacking inputs
2018-04-03 10-23-00 minerva >>> step classifier_network loading...
2018-04-03 10-23-01 minerva >>> step classifier_network transforming...
100%|██████████| 16/16 [00:35<00:00,  2.24s/it]
2018-04-03 10-23-37 minerva >>> step classifier_calibrator adapting inputs
2018-04-03 10-23-37 minerva >>> step classifier_calibrator loading...
2018-04-03 10-23-37 minerva >>> step classifier_calibrator transforming...
2018-04-03 10-23-37 minerva >>> step classifier_encoder adapting inputs
2018-04-03 10-23-37 minerva >>> step classifier_encoder loading...
2018-04-03 10-23-37 minerva >>> step classifier_encoder transforming...
2018-04-03 10-23-37 minerva >>> step classifier_output adapting inputs
2018-04-03 10-23-37 minerva >>> step classifier_output loading...
2018-04-03 10-23-37 minerva >>> step classifier_output transforming...
2018-04-03 10-23-37 minerva >>> step classifier_encoder adapting inputs
2018-04-03 10-23-37 minerva >>> step classifier_encoder loading...
2018-04-03 10-23-37 minerva >>> step classifier_encoder transforming...
2018-04-03 10-23-37 minerva >>> step classifier_loader adapting inputs
2018-04-03 10-23-37 minerva >>> step classifier_loader loading...
2018-04-03 10-23-37 minerva >>> step classifier_loader transforming...
2018-04-03 10-23-37 minerva >>> step classifier_network unpacking inputs
2018-04-03 10-23-37 minerva >>> step classifier_network loading...
2018-04-03 10-23-37 minerva >>> step classifier_network transforming...
100%|██████████| 15/15 [00:26<00:00,  1.74s/it]
2018-04-03 10-24-03 minerva >>> step classifier_calibrator adapting inputs
2018-04-03 10-24-03 minerva >>> step classifier_calibrator loading...
2018-04-03 10-24-03 minerva >>> step classifier_calibrator transforming...
2018-04-03 10-24-03 minerva >>> step classifier_encoder adapting inputs
2018-04-03 10-24-03 minerva >>> step classifier_encoder loading...
2018-04-03 10-24-03 minerva >>> step classifier_encoder transforming...
2018-04-03 10-24-03 minerva >>> step classifier_output adapting inputs
2018-04-03 10-24-03 minerva >>> step classifier_output loading...
2018-04-03 10-24-03 minerva >>> step classifier_output transforming...

Validation score is 2.1188
Test score is 2.0907
That is a solid validation
Sorry, but this score is not high enough to pass the task
Calculated experiment snapshot size: 9.08 MB

1.408208 | 2018-04-03 10-31-59 minerva >>> starting experiment...
-- | --
3.79994 | 2018-04-03 10-32-01 minerva >>> running: alignment
5.394486 | 2018-04-03 10-32-03 minerva >>> step aligner_encoder adapting inputs
5.394776 | 2018-04-03 10-32-03 minerva >>> step aligner_encoder loading...
5.394961 | 2018-04-03 10-32-03 minerva >>> step aligner_encoder transforming...
5.395138 | 2018-04-03 10-32-03 minerva >>> step aligner_loader adapting inputs
5.395315 | 2018-04-03 10-32-03 minerva >>> step aligner_loader loading...
5.39549 | 2018-04-03 10-32-03 minerva >>> step aligner_loader transforming...
5.395664 | 2018-04-03 10-32-03 minerva >>> step aligner_network unpacking inputs
5.395837 | 2018-04-03 10-32-03 minerva >>> step aligner_network loading...
7.896086 | 2018-04-03 10-32-05 minerva >>> step aligner_network transforming...
46.352937 | 2018-04-03 10-32-44 minerva >>> step aligner_unbinner unpacking inputs
46.353462 | 2018-04-03 10-32-44 minerva >>> step aligner_unbinner loading...
46.35368 | 2018-04-03 10-32-44 minerva >>> step aligner_unbinner transforming...
46.353893 | 2018-04-03 10-32-44 minerva >>> step aligner_adjuster adapting inputs
46.575793 | 2018-04-03 10-32-44 minerva >>> step aligner_adjuster loading...
46.576187 | 2018-04-03 10-32-44 minerva >>> step aligner_adjuster transforming...
46.576443 | 2018-04-03 10-32-44 minerva >>> step aligner_output adapting inputs
46.576672 | 2018-04-03 10-32-44 minerva >>> step aligner_output loading...
46.576923 | 2018-04-03 10-32-44 minerva >>> step aligner_output transforming...
46.577152 | 2018-04-03 10-32-44 minerva >>> step aligner_encoder adapting inputs
46.57738 | 2018-04-03 10-32-44 minerva >>> step aligner_encoder loading...
46.57762 | 2018-04-03 10-32-44 minerva >>> step aligner_encoder transforming...
46.577851 | 2018-04-03 10-32-44 minerva >>> step aligner_loader adapting inputs
46.578076 | 2018-04-03 10-32-44 minerva >>> step aligner_loader loading...
46.578317 | 2018-04-03 10-32-44 minerva >>> step aligner_loader transforming...
46.57855 | 2018-04-03 10-32-44 minerva >>> step aligner_network unpacking inputs
46.578779 | 2018-04-03 10-32-44 minerva >>> step aligner_network loading...
46.579016 | 2018-04-03 10-32-44 minerva >>> step aligner_network transforming...
74.03968 | 2018-04-03 10-33-11 minerva >>> step aligner_unbinner unpacking inputs
74.03987 | 2018-04-03 10-33-11 minerva >>> step aligner_unbinner loading...
74.040058 | 2018-04-03 10-33-11 minerva >>> step aligner_unbinner transforming...
74.040246 | 2018-04-03 10-33-11 minerva >>> step aligner_adjuster adapting inputs
74.040454 | 2018-04-03 10-33-11 minerva >>> step aligner_adjuster loading...
74.040641 | 2018-04-03 10-33-11 minerva >>> step aligner_adjuster transforming...
74.040828 | 2018-04-03 10-33-11 minerva >>> step aligner_output adapting inputs
74.041018 | 2018-04-03 10-33-11 minerva >>> step aligner_output loading...
74.041203 | 2018-04-03 10-33-11 minerva >>> step aligner_output transforming...
74.041387 |  
74.041575 | Validation score is 62.5229
74.04176 | Test score is 64.9918
74.041945 | That is a solid validation
74.042129 | Congrats you solved the task!
74.042313 | 2018-04-03 10-33-11 minerva >>> running: localization
75.068115 | 2018-04-03 10-33-12 minerva >>> step localizer_loader unpacking inputs
75.068325 | 2018-04-03 10-33-12 minerva >>> step localizer_loader loading...
75.068492 | 2018-04-03 10-33-12 minerva >>> step localizer_loader transforming...
75.068652 | 2018-04-03 10-33-12 minerva >>> step localizer_network unpacking inputs
75.068768 | 2018-04-03 10-33-12 minerva >>> step localizer_network loading...
75.068882 | 2018-04-03 10-33-12 minerva >>> step localizer_network transforming...
101.504964 | 2018-04-03 10-33-39 minerva >>> step localizer_unbinner unpacking inputs
101.505176 | 2018-04-03 10-33-39 minerva >>> step localizer_unbinner loading...
101.505376 | 2018-04-03 10-33-39 minerva >>> step localizer_unbinner transforming...
101.505558 | 2018-04-03 10-33-39 minerva >>> step localizer_output adapting inputs
101.505748 | 2018-04-03 10-33-39 minerva >>> step localizer_output loading...
101.505929 | 2018-04-03 10-33-39 minerva >>> step localizer_output transforming...
101.506116 | 2018-04-03 10-33-39 minerva >>> step localizer_loader unpacking inputs
101.506312 | 2018-04-03 10-33-39 minerva >>> step localizer_loader loading...
101.506507 | 2018-04-03 10-33-39 minerva >>> step localizer_loader transforming...
101.506699 | 2018-04-03 10-33-39 minerva >>> step localizer_network unpacking inputs
101.506881 | 2018-04-03 10-33-39 minerva >>> step localizer_network loading...
101.50707 | 2018-04-03 10-33-39 minerva >>> step localizer_network transforming...
128.154553 | 2018-04-03 10-34-06 minerva >>> step localizer_unbinner unpacking inputs
128.154753 | 2018-04-03 10-34-06 minerva >>> step localizer_unbinner loading...
128.154942 | 2018-04-03 10-34-06 minerva >>> step localizer_unbinner transforming...
128.155128 | 2018-04-03 10-34-06 minerva >>> step localizer_output adapting inputs
128.15532 | 2018-04-03 10-34-06 minerva >>> step localizer_output loading...
128.155509 | 2018-04-03 10-34-06 minerva >>> step localizer_output transforming...
128.155701 |  
128.155905 | Validation score is 108.2913
128.156099 | Test score is 94.9431
128.156303 | That is a solid validation
128.156496 | Congrats you solved the task!
128.156686 | 2018-04-03 10-34-06 minerva >>> running: classification
129.684731 | 2018-04-03 10-34-07 minerva >>> step classifier_encoder adapting inputs
129.684952 | 2018-04-03 10-34-07 minerva >>> step classifier_encoder loading...
129.685101 | 2018-04-03 10-34-07 minerva >>> step classifier_encoder transforming...
129.685222 | 2018-04-03 10-34-07 minerva >>> step classifier_loader adapting inputs
129.685368 | 2018-04-03 10-34-07 minerva >>> step classifier_loader loading...
129.685489 | 2018-04-03 10-34-07 minerva >>> step classifier_loader transforming...
129.685635 | 2018-04-03 10-34-07 minerva >>> step classifier_network unpacking inputs
129.68579 | 2018-04-03 10-34-07 minerva >>> step classifier_network loading...
129.685929 | 2018-04-03 10-34-07 minerva >>> step classifier_network transforming...
158.137679 | 2018-04-03 10-34-36 minerva >>> step classifier_calibrator adapting inputs
158.138025 | 2018-04-03 10-34-36 minerva >>> step classifier_calibrator loading...
158.138231 | 2018-04-03 10-34-36 minerva >>> step classifier_calibrator transforming...
158.138611 | 2018-04-03 10-34-36 minerva >>> step classifier_encoder adapting inputs
158.138809 | 2018-04-03 10-34-36 minerva >>> step classifier_encoder loading...
158.138997 | 2018-04-03 10-34-36 minerva >>> step classifier_encoder transforming...
158.139212 | 2018-04-03 10-34-36 minerva >>> step classifier_output adapting inputs
158.139738 | 2018-04-03 10-34-36 minerva >>> step classifier_output loading...
158.139933 | 2018-04-03 10-34-36 minerva >>> step classifier_output transforming...
158.140115 | 2018-04-03 10-34-36 minerva >>> step classifier_encoder adapting inputs
158.140311 | 2018-04-03 10-34-36 minerva >>> step classifier_encoder loading...
158.140506 | 2018-04-03 10-34-36 minerva >>> step classifier_encoder transforming...
158.140697 | 2018-04-03 10-34-36 minerva >>> step classifier_loader adapting inputs
158.14089 | 2018-04-03 10-34-36 minerva >>> step classifier_loader loading...
158.141069 | 2018-04-03 10-34-36 minerva >>> step classifier_loader transforming...
158.141254 | 2018-04-03 10-34-36 minerva >>> step classifier_network unpacking inputs
158.141435 | 2018-04-03 10-34-36 minerva >>> step classifier_network loading...
158.347917 | 2018-04-03 10-34-36 minerva >>> step classifier_network transforming...
196.301351 | 2018-04-03 10-35-14 minerva >>> step classifier_calibrator adapting inputs
196.301537 | 2018-04-03 10-35-14 minerva >>> step classifier_calibrator loading...
196.301715 | 2018-04-03 10-35-14 minerva >>> step classifier_calibrator transforming...
196.301893 | 2018-04-03 10-35-14 minerva >>> step classifier_encoder adapting inputs
196.302067 | 2018-04-03 10-35-14 minerva >>> step classifier_encoder loading...
196.302249 | 2018-04-03 10-35-14 minerva >>> step classifier_encoder transforming...
196.302466 | 2018-04-03 10-35-14 minerva >>> step classifier_output adapting inputs
196.302687 | 2018-04-03 10-35-14 minerva >>> step classifier_output loading...
196.302904 | 2018-04-03 10-35-14 minerva >>> step classifier_output transforming...
196.505624 |  
196.505941 | Validation score is 2.1188
196.506135 | Test score is 2.0907
196.506323 | That is a solid validation
196.506537 | Sorry, but this score is not high enough to pass the task

[rafajak]

Small correction - I tried replicating the above, but this time the program didn't terminate immediately after failed classification, but went on to complete (successfully) localization and alignment tasks.