Is logits_bias trainable?

[audreycs]

Hello, I am confused about the variable logits_bias. It seems that you make logits_bias not trainable in your code. (At line 170 in memn2n_kv.py) https://github.com/siyuanzhao/automated-essay-grading/blob/cda24d8522c2faeb919719de002cd2754cbf1973/memn2n_kv.py#L170 Normally, shouldn't bias be trainable?

Other possible mistakes I found: 1. https://github.com/siyuanzhao/automated-essay-grading/blob/cda24d8522c2faeb919719de002cd2754cbf1973/cv_train.py#L191 Here sent_size_list is the list of all data(train+test), but score_idx is the index of train data.

https://github.com/siyuanzhao/automated-essay-grading/blob/cda24d8522c2faeb919719de002cd2754cbf1973/memn2n_kv.py#L177 why here logits are used to calculate cross entropy, but not the softmax result of logits.

Thanks!