devolearn icon indicating copy to clipboard operation
devolearn copied to clipboard

tests: unit tests do not check the outputs of models

Open Mayukhdeb opened this issue 4 years ago • 2 comments

While the unit tests do cover the output types, they do not check if the outputs themselves are correct/incorrect.

To do:

  • Use some loss function w.r.t ideal outputs and make sure the losses are low enough
  • Make sure the output dtype matches the original (int, float32, float64)

Mayukhdeb avatar Apr 02 '21 07:04 Mayukhdeb

Hello @Mayukhdeb, for the lineage population model we can calculate loss from the ground truth population labels from the CSV. But, I don't understand how can we test the output of the segmentation model. One way I think could be running the model on the same image multiple times and calculating the mutual loss between them. If that would be helpful, I wish to work on the issue.

vrutikrabadia avatar Apr 02 '21 08:04 vrutikrabadia

Hi @vrutikrabadia.

For the segmentation model, I plan to execute the testing procedure by saving a few .npy files which would then be compared with the predictions with np.allclose().

Mayukhdeb avatar Apr 02 '21 08:04 Mayukhdeb