Does loss curves look right?

[yinguoxiangyi]

The following is the loss curve of my own dataset. Does it look right? it seems that binary code loss didn't decrease.

[suyz526]

The binary code loss is only calculated for pixels within the predicted mask.

For debugging, you can set mask_binary_code_loss to False to check if the binary code loss converges.

[yinguoxiangyi]

According to your past experiments, ideally, which value the binary_code_loss will decrease to ?

[suyz526]

I don't have any logs available at this moment. Is your object symmetrical?

[jonashein]

Hi @yinguoxiangyi ,

I observed the same behaviour when training Zebrapose on my custom dataset:

mask_binary_code_loss = False results in a nice exponentially decreasing curve:

mask_binary_code_loss = True results in a somewhat constant graph at first:

I continued the training for a bit longer (with mask_binary_code_loss = True), and at some point the binary code loss starts to decrease. Though note the different scale of the y axis compared to the previous screenshot:

The test-time metrics are increasing, so it seems like the training is running just fine, despite the unusual looking graph.