Compatibility with Tensorflow 1.14

[ales004]

When I tried to train the deeptype model using Tensorflow 1.14 it failed.

On this line it failed saying that it got the input_mode argument twice. After some digging in Tensorflow versions, I think that the problem lies in these imports: https://github.com/openai/deeptype/blob/11568183dd67a472cddbc286690a6153c07710b6/learning/train_type.py#L20-L31

because in the latest Tensorflow, the imports in the first part are all present but they point to another (newer) implement of the classes. What worked for me was to replace the lines with:

RNNCell = tf.contrib.rnn.RNNCell
TFLSTMCell = tf.contrib.rnn.LSTMCell
MultiRNNCell = tf.contrib.rnn.MultiRNNCell
LSTMStateTuple = tf.contrib.rnn.LSTMStateTuple

I also had to replace these lines: https://github.com/openai/deeptype/blob/11568183dd67a472cddbc286690a6153c07710b6/learning/train_type.py#L51-L52

with this code:

        _beta1_power, _beta2_power = self._get_beta_accumulators()
        beta1_power = tf.cast(_beta1_power, var.dtype.base_dtype)
        beta2_power = tf.cast(_beta2_power, var.dtype.base_dtype)

[ottokart]

I also had to change: from tensorflow.contrib.cudnn_rnn import CudnnLSTM to: from tensorflow.contrib.cudnn_rnn.python.ops.cudnn_rnn_ops import CudnnLSTM in train_type.py line 25.

[JonathanRaiman]

@ottokart Good catch. I think some of this code is also superseded by a TF CPU Cudnn cell which might be better but I haven't tried it out yet (this codebase has its own CPU Cudnn cell)