Parameter initialization is wrong.

[ljpadam]

In the network.py:

nn.init.constant_(self.pos_conv.bias, 0)
nn.init.constant_(self.reg_conv.bias, -math.log(0.99/0.01))

This causes a large loss at the beginning.

These two lines should be amended as:

nn.init.constant_(self.pos_conv.bias,  -math.log(0.99/0.01))
nn.init.constant_(self.reg_conv.bias, 0)

[lwpyr]

yes, you are right. But a very interesting thing is the wrong initialization doesn't harm the result, though the loss is large at the beginning. This is a finding in my experiment, which is a little bit wired.