CNTK Crash: No computation node mapping exists for Variable Placeholder

[maybeLee]

When I am using keras with CNTK backend to load and predict on a LSTM model, CNTK crash with the following bug trace:

About to throw exception 'Function 'Composite(Combine): Input('lstm_1_copy_LA_copy_LA_copy_LA_copy_CP_input', [#], [49 x 1]) -> Output('Plus1481_Output_0', [#], [1])': No computation node mapping exists for Variable Placeholder('Placeholder1594', [#, toSequence_Minus537_Output_0], [100]).'
Traceback (most recent call last):
  File "exp1/job0/scripts/generation/script_prediction.py", line 121, in <module>
    _get_prediction(bk=bk, model_path=flags.model_path, batch_size=batch_size)
  File "exp1/job0/scripts/generation/script_prediction.py", line 42, in _get_prediction
    pred = model.predict(x,batch_size=batch_size)
  File "lib/python3.6/site-packages/keras/engine/training.py", line 1169, in predict
    steps=steps)
  File "lib/python3.6/site-packages/keras/engine/training_arrays.py", line 294, in predict_loop
    batch_outs = f(ins_batch)
  File "lib/python3.6/site-packages/keras/backend/cntk_backend.py", line 2016, in __call__
    output_values = self.metrics_func.eval(input_dict, as_numpy=False)
  File "cntk/cntk/bindings/python/cntk/ops/functions.py", line 733, in eval
    _, output_map = self.forward(arguments, outputs, device=device, as_numpy=as_numpy)
  File "cntk/cntk/bindings/python/cntk/internal/swig_helper.py", line 69, in wrapper
    result = f(*args, **kwds)
  File "cntk/cntk/bindings/python/cntk/ops/functions.py", line 867, in forward
    keep_for_backward)
  File "cntk/cntk/bindings/python/cntk/cntk_py.py", line 1980, in _forward
    return _cntk_py.Function__forward(self, *args)
RuntimeError: Function 'Composite(Combine): Input('lstm_1_copy_LA_copy_LA_copy_LA_copy_CP_input', [#], [49 x 1]) -> Output('Plus1481_Output_0', [#], [1])': No computation node mapping exists for Variable Placeholder('Placeholder1594', [#, toSequence_Minus537_Output_0], [100]).

Steps to reproduce: Please access the json configuration of the used lstm model from: https://drive.google.com/file/d/1BwXQEpnmutqW1sQOUhTP1e6StlqMh9uU/view?usp=sharing save the configuration file as model.json and then use the following script to reproduce bug:

import os
import argparse
import sys
import warnings
import configparser
import redis
import pickle


if __name__ == "__main__":
    parse = argparse.ArgumentParser()
    parse.add_argument("--backend", type=str, help="specify the backend")
    flags, _ = parse.parse_known_args(sys.argv[1:])

    bk = flags.backend
    os.environ["KERAS_BACKEND"] = bk
    os.environ["CUDA_VISIBLE_DEVICES"] = "0,1"
    warnings.filterwarnings("ignore", category=DeprecationWarning)
    warnings.filterwarnings("ignore", category=UserWarning)
    warnings.filterwarnings("ignore", category=FutureWarning)
    warnings.filterwarnings("ignore")

    if bk == "cntk":
        from cntk.device import try_set_default_device, gpu
        try_set_default_device(gpu(0))
    import numpy as np
    import keras
    model_path = "model.json"
    print("start loading the model")
    model = keras.models.model_from_json(open(model_path, "r").read())
    model.summary()
    input_shape = model.layers[0].input_shape
    input_shape = list(input_shape)
    input_shape[0] = 3
    input_shape = tuple(input_shape)
    x = np.random.rand(*input_shape)
    pred = model.predict(x)
    json = model.to_json()
    print(pred)
    print("successfully get the prediction result")
    del model

Note that when I use the different backend of Keras such as TensorFlow and Theano, this model can be successfully loaded and predicted.