InvalidArgumentError when running squad_seq2seq_v2_train.py on train-v1.1 data set

[amindahma]

`{'num_input_question_tokens': 3989, 'num_input_paragraph_tokens': 5002, 'num_target_tokens': 3429, 'input_paragraph_max_seq_length': 298, 'input_question_max_seq_length': 32, 'target_max_seq_length': 36} 2077 520 /usr/local/lib/python3.6/dist-packages/keras/engine/topology.py:2361: UserWarning: Layer decoder_lstm was passed non-serializable keyword arguments: {'initial_state': [<tf.Tensor 'encoder_lstm/while/Exit_2:0' shape=(?, 256) dtype=float32>, <tf.Tensor 'encoder_lstm/while/Exit_3:0' shape=(?, 256) dtype=float32>]}. They will not be included in the serialized model (and thus will be missing at deserialization time). str(node.arguments) + '. They will not be included ' Epoch 1/200 2018-06-10 08:35:38.892502: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

1/32 [..............................] - ETA: 3:50 - loss: 0.8656 - acc: 0.0000e+00 2/32 [>.............................] - ETA: 3:29 - loss: 0.9127 - acc: 0.0137
3/32 [=>............................] - ETA: 3:16 - loss: 0.9194 - acc: 0.0184 4/32 [==>...........................] - ETA: 3:08 - loss: 0.9289 - acc: 0.0207 5/32 [===>..........................] - ETA: 2:59 - loss: 0.8998 - acc: 0.0221 6/32 [====>.........................] - ETA: 2:51 - loss: 0.8553 - acc: 0.0231 7/32 [=====>........................] - ETA: 2:44 - loss: 0.8253 - acc: 0.0237 8/32 [======>.......................] - ETA: 2:36 - loss: 0.8145 - acc: 0.0243 9/32 [=======>......................] - ETA: 2:30 - loss: 0.7996 - acc: 0.0246 10/32 [========>.....................] - ETA: 2:23 - loss: 0.7879 - acc: 0.0250 11/32 [=========>....................] - ETA: 2:16 - loss: 0.7720 - acc: 0.0252 12/32 [==========>...................] - ETA: 2:10 - loss: 0.7623 - acc: 0.0254 13/32 [===========>..................] - ETA: 2:03 - loss: 0.7605 - acc: 0.0256 14/32 [============>.................] - ETA: 1:56 - loss: 0.7549 - acc: 0.0258 15/32 [=============>................] - ETA: 1:50 - loss: 0.7592 - acc: 0.0259 16/32 [==============>...............] - ETA: 1:43 - loss: 0.7631 - acc: 0.0260 17/32 [==============>...............] - ETA: 1:36 - loss: 0.7630 - acc: 0.0261 18/32 [===============>..............] - ETA: 1:30 - loss: 0.7544 - acc: 0.0262 19/32 [================>.............] - ETA: 1:23 - loss: 0.7663 - acc: 0.0263 20/32 [=================>............] - ETA: 1:17 - loss: 0.7661 - acc: 0.0264 21/32 [==================>...........] - ETA: 1:11 - loss: 0.7688 - acc: 0.0264 22/32 [===================>..........] - ETA: 1:04 - loss: 0.7708 - acc: 0.0265Traceback (most recent call last): File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call return fn(*args) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn status, run_metadata) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[19,27] = 3989 is not in [0, 3989) [[Node: question_embedding/Gather = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](question_embedding/embeddings/read, question_embedding/Cast)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/PycharmProjects/keras-question-and-answering-web-api/demo/squad_seq2seq_v2_train.py", line 22, in main() File "/opt/PycharmProjects/keras-question-and-answering-web-api/demo/squad_seq2seq_v2_train.py", line 18, in main random_state=random_state) File "/opt/PycharmProjects/keras-question-and-answering-web-api/keras_question_and_answering_system/library/seq2seq_v2.py", line 250, in fit callbacks=[checkpoint]) File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 2177, in fit_generator class_weight=class_weight) File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1849, in train_on_batch outputs = self.train_function(ins) File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 2475, in call **self.session_kwargs) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 889, in run run_metadata_ptr) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1120, in _run feed_dict_tensor, options, run_metadata) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run options, run_metadata) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[19,27] = 3989 is not in [0, 3989) [[Node: question_embedding/Gather = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](question_embedding/embeddings/read, question_embedding/Cast)]]

Caused by op 'question_embedding/Gather', defined at: File "/opt/PycharmProjects/keras-question-and-answering-web-api/demo/squad_seq2seq_v2_train.py", line 22, in main() File "/opt/PycharmProjects/keras-question-and-answering-web-api/demo/squad_seq2seq_v2_train.py", line 18, in main random_state=random_state) File "/opt/PycharmProjects/keras-question-and-answering-web-api/keras_question_and_answering_system/library/seq2seq_v2.py", line 234, in fit self.create_model() File "/opt/PycharmProjects/keras-question-and-answering-web-api/keras_question_and_answering_system/library/seq2seq_v2.py", line 113, in create_model name='question_embedding')(question_inputs) File "/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py", line 617, in call output = self.call(inputs, **kwargs) File "/usr/local/lib/python3.6/dist-packages/keras/layers/embeddings.py", line 138, in call out = K.gather(self.embeddings, inputs) File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 1208, in gather return tf.gather(reference, indices) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 2486, in gather params, indices, validate_indices=validate_indices, name=name) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1834, in gather validate_indices=validate_indices, name=name) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op op_def=op_def) File "/home/aminda/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): indices[19,27] = 3989 is not in [0, 3989) [[Node: question_embedding/Gather = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](question_embedding/embeddings/read, question_embedding/Cast)]] `