Compatibility with Tensorflow 1.14

[ales004]

When I tried to train the deeptype model using Tensorflow 1.14 it failed.

On this line it failed saying that it got the input_mode argument twice. After some digging in Tensorflow versions, I think that the problem lies in these imports:

deeptype/learning/train_type.py

Lines 20 to 31 in 1156818

	try:
	RNNCell = tf.nn.rnn_cell.RNNCell
	TFLSTMCell = tf.nn.rnn_cell.LSTMCell
	MultiRNNCell = tf.nn.rnn_cell.MultiRNNCell
	LSTMStateTuple = tf.nn.rnn_cell.LSTMStateTuple
	from tensorflow.contrib.cudnn_rnn import CudnnLSTM
	except AttributeError:
	RNNCell = tf.contrib.rnn.RNNCell
	TFLSTMCell = tf.contrib.rnn.LSTMCell
	MultiRNNCell = tf.contrib.rnn.MultiRNNCell
	LSTMStateTuple = tf.contrib.rnn.LSTMStateTuple
	from tensorflow.contrib.cudnn_rnn.python.ops.cudnn_rnn_ops import CudnnLSTM

because in the latest Tensorflow, the imports in the first part are all present but they point to another (newer) implement of the classes. What worked for me was to replace the lines with:

RNNCell = tf.contrib.rnn.RNNCell
TFLSTMCell = tf.contrib.rnn.LSTMCell
MultiRNNCell = tf.contrib.rnn.MultiRNNCell
LSTMStateTuple = tf.contrib.rnn.LSTMStateTuple

I also had to replace these lines:

deeptype/learning/train_type.py

Lines 51 to 52 in 1156818

	beta1_power = tf.cast(self._beta1_power, var.dtype.base_dtype)
	beta2_power = tf.cast(self._beta2_power, var.dtype.base_dtype)

with this code:

        _beta1_power, _beta2_power = self._get_beta_accumulators()
        beta1_power = tf.cast(_beta1_power, var.dtype.base_dtype)
        beta2_power = tf.cast(_beta2_power, var.dtype.base_dtype)

[ottokart]

I also had to change:
from tensorflow.contrib.cudnn_rnn import CudnnLSTM
to:
from tensorflow.contrib.cudnn_rnn.python.ops.cudnn_rnn_ops import CudnnLSTM
in train_type.py line 25.

[JonathanRaiman]

@ottokart Good catch. I think some of this code is also superseded by a TF CPU Cudnn cell which might be better but I haven't tried it out yet (this codebase has its own CPU Cudnn cell)