Probability of one or both classes tend to get stuck at exactly 50%

[Snubben-B]

Hi, I'm trying to classify pixels into one of two categories. I have a problem that the probability of one or both classes tend to get stuck at exactly 50%. It does not happen every time, but very often. I have encountered this problem for several different settings of the hyperparameters and both for adam and momentum. I use cross entropy as the cost function. This occurs even in the very simple case where I have a binary blob as input, and I want the same blob as output. What could cause this? I'm attaching an image of the issue. For this specific case I used layers=5, and features_root=64.

[jakeret]

Hard to tell what is causing this. Have you tried to normalize your input?

[Snubben-B]

I'm not entirely sure of what you mean with normalization in this case. The input is binary, with the values 0 and 1.

[jakeret]

I was talking about removing the mean and scaling to unit variance.

I'm not sure if I understand your case. You have binary input data and you want to learn a binary classification?

[Snubben-B]

To clarify, I initially had a more complex case, in which I experienced the same problem. The binary input -> binary output is a simplified case, but the problem persists.