The optimization objective is the categorical_crossentropy？

[raymon-tian]

Hi, I realize that the optimization objective is the categorical_crossentropy in your code, i.e., model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']). However, in my opinion, if we want to train the neural network with the IB, the IB objective should be set as the loss, i.e., model.compile(loss='IB_loss ', optimizer='adam', metrics=['accuracy']). Or maybe the IB just acts as a regularizer?

I am confused about it. Could you make an explanation about it? thanks.

Best

[artemyk]

The cross-entropy loss acts as a variational lower bound on the I(M;Y) term. The compression term I(M;X) is added in as a regularizer.